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EXECUTIVE SUMMARY 

This study investigated the relationships between (a) two dimensions of adult literacy 
instruction and (b) change in the literacy practices of adult literacy students. The two 
instructional dimensions investigated were (a) degree of authenticity of the activities and texts 
employed in the literacy class and (b) degree of teacher/student collaboration around activities, 
texts, assessments, and program governance. The construct of authenticity for this study was 
defined as those literacy activities and purposes used by people in their lives, excluding those that 
are structured solely around learning to read and write in school settings. 

The study was motivated by issues of intergenerational literacy and by research that 
documents that young children’s success and failure in learning to read and write in school is 
related to the frequency of literacy events and types of texts read and written by their parents. It 
is assumed that increasing the frequencies and types of literacy practices of adult literacy students 
is an important outcome of adult instruction due to this intergenerational literacy factor. 

Data on the two instructional dimensions of interest - authenticity and collaboration - was 
collected from 83 adult literacy classes in 22 states. Data on class activities and texts and on 
degree of student influence on these were triangulated from three sources: a teacher 
questionnaire; class observation; and group-student interview. Classes were given a score 
indicating the degree to which they reflected these two dimensions. 

Data on change in adult students’ literacy practices was collected from 173 adult literacy 
students with a detailed questionnaire that was administered to students individually in their 
homes every three months for up to a year, as long as they attended their literacy class. Students 
were asked if they engaged in individual literacy practices and if so were these practices new or 
engaged in more frequently since beginning the class. 

Data were analyzed using Hierarchical Linear Modeling to model change. Due to problems 
with missing data, the results of the analysis were based on 157 students in 77 adult literacy 
classes. The questionnaire responses were placed on a common scale using Item Response 
Theory from which a change score was derived. Using this outcome variable, the effects of the 
two instructional dimensions on change were modeled, controlling for the following student- 
level variables: literacy level of the student on entry to the class; days students had attended the 
class; attendance pattern of student; ESOL status of student; gender; whether the student was in a 
class or had a one-on-one tutor. Class-level variables controlled for were the types of classes 
ESOL, ABE, and Family Literacy. 
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Results of the analysis revealed that authenticity of class literacy activities and texts had a 
moderate effect on change in student literacy practices, operationalized as increases in frequency 
of reading and writing and/or types of texts read and written. Analysis of the literacy engagement 
and change scales revealed that the increases in types of texts involved reading and writing more 
texts at higher levels of discourse, levels associated with higher levels of emergent literacy 
knowledge in previous studies. The degree of collaboration between teacher and students 
showed no effect on literacy practice change. 

Student-level factors that also showed independent effects on change in literacy practice 
were (a) literacy level of student at entry to class; (b) number of days the student had been 
attending the class; and (c) non-ESOL status of student. Besides authenticity, the other 
classroom-level factor to show a significant effect on literacy practice change was ABE format. 

A qualitative analysis was done using the spontaneous student comments on 173 home 
literacy practices questionnaires attributing changes in literacy practice. Results showed that 
students contextualized their reported literacy practice changes to life changes such as changes in 
employment, changes emanating from learning to read and write, changes in living situations, 
and family situations. These results demonstrated the socially-situated nature of literacy and 
literacy change. 

The results of the quantitative analysis are taken to suggest ways that adult literacy 
instruction can help to change the cycle of intergenerational literacy success and failure. As 
such, the significance of the results go beyond the field of adult literacy alone. Nevertheless, the 
results of this study demonstrate that research can document instructional outcome in ways that 
reflect actualized literacy learning in sociocultural contexts. 
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AFFECTING CHANGE IN LITERACY PRACTICES 
OF ADULT LEARNERS: 

IMPACT OF TWO DIMENSIONS OF INSTRUCTION 
CHAPTER 1: THE STUDY OF ADULT LITERACY PRACTICE 

When adult learners go to school to learn to read and write better, what do they do 
with this learning? Do they read and write more in their out-of-school lives? Do they 
begin to read and write different kinds of texts in their out-of-school lives? If such changes 
in literacy practices occur for adult literacy students, which dimensions of their literacy 
instruction can we suggest are related to changes such as these, if they do occur? 

This study examines two aspects of instruction that could be related to changes in 
the types and frequencies with which adult literacy learners engage in reading and writing 
activities in their lives outside of school. Its significance lies in a broader view of adult 
literacy that sees adults’ literacy practices as part of an intergenerational cycle of literacy 
and intimately tied to the emergent literacy development of children. That is, children’s 
development of emergent literacy concepts is dependent upon their experience with literate 
others in their lives using print for real-life reasons (Purcell-Gates, 1995, 1996; Purcell- 
Gates & Dahl, 1991; Teale & Sulzby, 1986). Thus, change in literacy practices of adult 
learners also affects the emergent literacy development of the young children connected to 
the adult learners, and increases in emergent literacy knowledge is positively related to 
success in learning to read and write in school (Purcell-Gates & Dahl, 1991; Snow, Bums, 
& Griffin, 1998). 

This study also contributes to a social practice theory of literacy (Barton & 
Hamilton, 1998) which views literacy as best understood as a set of social practices which 
are associated with different domains of life and which are purposeful and embedded in 
broader social goals and cultural practices. The questions pursued by this study explore the 
relationships, if any, between these social and cultural practices and adult literacy 
instruction. In other words, with this study, we began to explore what we refer to as the 
actualization of literacy instruction: the playing out in practice what has been attended to 
by focused formal instruction. While the major outcomes of the study result from 
quantitative analysis, we also include the participants voices as they reflect their 
experiences of literacy instruction and their literacy practices, with a resulting qualitative 
lens on the literacy practice change. 

Given the above, this study can be viewed as a measure of instructional outcome for 
adult literacy students. Rather than conducting the traditional outcome -assessment of 
change in literacy skill, or achievement, as measured by norm-referenced or criterion- 
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referenced tests (with all of the attendant difficulties with validity and reliability, especially 
for adult learners), we have assessed directly the impact of targeted aspects of instruction 
on the ultimate end-goal of all literacy instruction — actual reading and writing practices of 
the students. One way to think of this is to accept that the assessment of reading and 
writing skill level is an interim measure of instructional outcome, since no one would argue 
that a final test score is the instructional end-point. Rather, it must be used to infer actual 
use of that reading and writing skill, and it is this use, or practice of reading and writing 
that is the ultimate outcome, or goal, of literacy instruction. 

The study of adult learners' literacy practices is further motivated by the cognitive 
argument that literacy skills — as is true of all skills — must be practiced to become 
automatic, and that this practice is achieved in out-of-school contexts to a large degree by 
increased frequencies of reading and writing and/or reading and writing different types of 
texts in addition to those already commonly written and read. Automaticity (LaBerge & 
Samuels, 1974) in reading refers to fluent and effortless — in the sense of not requiring 
conscious attention — word recognition. Readers must attain automaticity in word 
recognition before they can attend to the actual purpose of reading — comprehension. The 
cognitive demands of reading and writing words change according to their written contexts, 
and it cannot be assumed that automatic recognition of a word in one context ensures 
automatic recognition in another. Written contexts reflect variation in genre, textual form, 
and syntax. Examining the different types of textual contexts inherent in different reading 
and writing practices (e.g. reading lists, novels, mortgages; writing essays, personal letters, 
notes) is thus required to explore the degree to which adult learners are practicing their 
newly acquired literacy skills with the implication that an outcome of this activity is 
increased automaticity of written language processing. 

Finally, this study was motivated by the need to empirically explore the theoretical 
claims, or inferences, that particular types of adult literacy instruction are better than others 
and that the benefits can be seen in the ways adults actually apply and use their reading and 
writing abilities. 



Theoretical Frame 

This study was conducted within a sociocultural theory of literacy development that 
views literacy as situated within sociocultural contexts and literacy use as cultural practice. 
Within this, the term literacy culture can be used to reflect all of the ways in which persons 
— within individual families and homes as well as within different sociocultural 
communities — use print, practice literacy, or engage in literacy practices. This implies the 
construct of multiple literacies, socially situated (Street, 1989). It also implies the social 
practice theory of literacy described above (Barton & Hamilton, 1998). 
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It must be noted that the definition of literacy practice that we assume differs 
slightly from that put forth by Barton and Hamilton in their ethnography of literacy practice 
in Lancaster, England, Local Literacies: Reading and Writing in One Community (1998, 
Cambridge University Press). Barton and Hamilton, coming from a social theory 
perspective, define literacy practices as 

. . .the general cultural ways of utilising written language which people draw 
upon in their lives. In the simplest sense literacy practices are what people 
do with literacy. However, practices are not observable units of behaviour 
since they also involve values, attitudes, feelings and social 
relationships.... This includes people’s awareness of literacy, constructions 
of literacy and discourses of literacy, how people talk about and make sense 
of literacy. These are processes internal to the individual; at the same time, 
practices are the social processes which connect people with one another, 
and they include shared cognitions represented in ideologies and social 
identities (pg. 6-7). 

Barton and Hamilton state that while literacy practices are unobservable, the 
associated literacy events and texts are observable units. Literacy events, for Barton and 
Hamilton, are activities where literacy has a role and include both texts and talk around 
those texts. 

While finding no fault with this definition of literacy practices and literacy events, 
we have chosen to focus on the central part of these: texts and the reading and writing of 
these texts. This differs partly because while Barton and Hamilton come from a social 
theory perspective, the senior author of this report comes from a sociopsycholinguistic 
perspective. As is described below, the study is motivated by interest in the cognitive and 
linguistic effects of different types of written texts on literacy learning. Thus, the focus is 
centered on texts that are involved in uses of written language. Therefore, when we use the 
term literacy practices we are referring to the reading and writing of specific texts for 
socially-situated purposes and intents. 
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CHAPTER TWO: RELATED LITERATURE 
Family Literacy and Emergent Literacy Development 

There is a wide body of research that documents the positive academic effects 
that accrue to young children who experience many and broad uses of print in their 
family lives (Snow, Bums, & Griffin, 1998). While a majority of this research 
focuses on the cognitive and linguistic benefits of reading to young children, other 
literacy practices also have been shown to relate positively to emergent literacy 
knowledge. Purcell-Gates (1996) documented significant relationships between the 
frequency of all types of reading and writing on children’s development of the basic 
emergent literacy concept that print is (a) linguistically meaningful and (b) functions 
in different ways in people’s lives. She also documented that the types of reading 
and writing practices engaged in by literate family members were significantly 
related to children’s development of knowledge about the alphabetic principle and an 
array of print-related concepts (Clay, 1985). 

Purcell-Gates and Dahl (1991), documented, in a two-year longitudinal study 
of low- SES kindergartners and first-graders, that those children who began school 
with greater knowledge of these same emergent literacy concepts were more 
successful at reading and writing at the end of first grade. In particular, the concept 
that print is linguistically meaningful and functional in people’s lives was the most 
predictive of success at learning to read and write in school. 

Adams (1990), in the landmark book, Beginning to Read , summarized 
hundreds of studies demonstrating that a necessary component of successful 
beginning reading was the ability to recognize and control the phonemic base of 
written English. This ability, widely known as phonemic awareness, is the precursor 
to being able to decode and encode words. The understanding that written English is 
encoded at the phonemic level 1 is what other emergent literacy researchers above, 
refer to as the alphabetic principle. Adams, as well as Snow, Bums, and Griffin 
(1999), synthesized a large body of research that demonstrated that phonemic 
awareness and knowledge of the alphabetic principle among young children was 
significantly related to both being read to in the home and to the overall literacy level 
and degree of reading and writing in the home environment. This latter was more 



'This roughly corresponds to the assignment of one sound, or phoneme, to one letter or letter unit. 
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directly demonstrated by the Purcell-Gates (1996) study of home literacy practices 
and young children’s emergent literacy knowledge. 

Taken as a whole, this research on family literacy practices and children’s 
development of emergent literacy concepts strongly suggests that changes in family 
literacy practices will affect emergent literacy development. This, in turn, will affect 
the entire cycle of intergenerational literacy achievement. Figure 1 depicts this cycle, 
as documented by research. 



Figure 1 . The home literacy culture/school literacy success cycle. 



Child School 
Success 



Home Literacy 




Theoretical Biases for Adult Literacy Instruction 

While very little empirical research exists on the outcomes of different types 
of adult literacy instruction (Wagner & Venezky, 1995), the prevailing belief among 
many academics and adult literacy program and policy leaders is that instruction 
which is (a) collaborative, dialogic, and responsive to the lives and needs of the 
learners (Auerbach, 1992; Freire, 1993; Horton in Glen, 1996; Purcell-Gates & 
Waterman, 2000) and (b) uses authentic, or real-life, literacy activities and materials 
is best for adult learners (Auerbach, 1995; Fingeret, 1991; Lytle, 1994; Stein, 2000). 
Thus, we focused on these two dimensions of adult literacy instruction in this study 
of the relationships between instruction and change in literacy practices. 
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Real-life Literacy as Part of Instruction 



As noted by Lytle (1994), adult literacy programs differ in their emphasis on 
teaching predetermined sets of skills, or alternately, in building the literacy practices 
of everyday life (p.5). This is reflected in the contents of adult literacy textbooks, 
which range from life skills and problem solving to phonics and word family drills. 
For this report, literacy work grounded in the life of the student outside of the 
classroom was considered real-life literacy . We also use the term authentic to 
describe these outside-of-school literacy practices and materials. 2 

The distinction between real-life literacy activities/materials and school-only 
appears to be important in light of research that has suggested that students learn 
most efficiently when instructional materials reflect and incorporate students’ prior 
experiences (Fingeret, 1991). Classroom activities using generative themes taken 
from the lives of adult learners have been seen to facilitate their acquisition of 
literacy (Freire, 1993; Glen, 1996; Purcell-Gates & Waterman, 2000). This 
perspective recognizes that adult learners have a wealth of experience on which to 
draw (Freire, 1993; U.S. Congress, Office of Technological Assistance, 1993). 

Given their many responsibilities (as parents and/or workers), adult literacy students 
have a limited amount of time for attending classes and studying. These students 
desire skills that they can use in the current contexts of their lives (Freire, 1993; U.S. 
Congress, Office of Technological Assistance, 1993) and often express a desire to 
use materials geared towards their day-to-day experience as adults and parents 
(Nwakeze & Seiler, 1993). 

The use of real-life literacy materials and activities in adult literacy programs 
is supported by research that documents the powerful role of context in learning. For 
example, many workplace literacy programs teach literacy skills as they are needed 
within specific work contexts. Compared to programs that concentrated on more 
“general” literacy, adult programs that incorporated job-related materials were 
associated with larger increases in both job-related and general literacy (Sticht, 

1989). Other studies have concluded that much of the growth made by participants 



2 We recognize the debate over the term §authentic§ as it applies to literacy events. While we 
acknowledge that school activities such as workbook skill work can be viewed as §authentic§ within a 
particular frame of pedagogy, we use this term for this report to reflect literacy uses found in social 
contexts outside of schools. In other words, we consider as §authentic§ those literacy events that 
researchers (Purcell-Gates, 1996; Tcalc, 1986) have documented as mediating people§s social and 
cultural lives. 
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in general literacy programs is likely to be lost if recently learned skills are not 
applied, and thus practiced, in real-life situations (Brizius & Foster, 1987). 
Transferring skills, however, between contexts is extremely difficult and rarely 
accomplished by learners to the degree often assumed by educators (NCAL, 1995). 
Thus, it would make sense to involve learners in real-life literacy practices in the 
classroom to ease the transference problem. 

However, the concept of “real-life” can actually be decontextualized in ways 
that reduce the effectiveness of its inclusion in adult literacy programs. Once 
activities and materials are mass- produced and mass- prescribed, they become 
increasingly distanced from the lives of individual students. Given the diversity of 
life situations among adult learners, this could easily happen in the adult literacy 
classroom. For example, a thematic unit centered around the use of checkbooks 
would be considered a real-life activity mediated by print by most middle-class 
people. However, it would not be contextually relevant for students who do not have 
checking accounts, have never had checking accounts, and have no realistic plans for 
opening checking accounts in the near future (Lerche, 1985). 

One way some practitioners avoid this inappropriate use of real-life 
activities/materials is to respond to their individual students’ literacy needs and elicit 
student-generated, student-provided, or student-requested texts. Hunter and Harman 
(1985) concluded that maximum use of this type of material was associated with 
higher levels of student achievement. Other researchers have suggested that student 
writing based on their own lives is associated with increases in writing skills (Stascz, 
Schwartz & Weeden, 1994; D’Annunzio, 1994). 

Student/Teacher Power Relationships 

Dialogic, collaborative, educational practice is that which includes the student 
as a participant and partner in the goals, activities, and procedures of the class and 
program. This is in contrast to the more typical practice wherein students cede 
authority and power to the teacher (or underlying program structure) for decisions 
regarding their learning. Freire (1993) refers to this latter type of education as a 
“banking” model of education, where the student is the passive recipient of the 
teacher’s knowledge. This retains the students’ object status, according to Freire, 
and precludes real learning or any significant changes in the lives of the students. To 
be truly liberatory, Freire maintains, an education must begin with the solution of the 
teacher-student contradiction, by reconciling the poles of the contradiction so that 
both are simultaneously teachers and students (p. 53). 



ERIC 



7 



NCSALL Reports #17 



November 2000 



There has been little research with regards to the influence of the structure of 
literacy instruction on students’ acquisition of literacy (Lytle, 1994). However, this 
distinction between collaborative and teacher-directed power relationships appears to 
be important. Studies have shown that student learning is enhanced when students 
are active partners (U.S. Congress, Office of Technological Assistance, 1993) 
involved in decision making about their education program (Brizius & Foster, 1987). 

Purcell-Gates and Waterman (2000) attributed a great deal of the success of an adult 
women’s literacy class in rural El Salvador to the dialogic nature of the relationship 
between the teacher — Waterman — and the students. The women, all survivors of an 
oppressed childhood and a brutal civil war and all of whom were basically 
nonliterate at the inception of the class, became effective readers and writers, 
assuming positions of leadership that required literate activity in their community. 

Fingeret (1991) also calls for dialogic practice and notes that curriculum 
development/teaching depends upon a knowledge of students' cultures. In 
dialogic/collaborative practice, instructors can be educated by their students about the 
students’ culture and history. Given the variety of cultures, many of them 
nonmainstream and/or immigrant, from which adult education students come, 
becoming educated about students’ cultures and histories may be a crucial element in 
adult education teaching in the United States at this time (NCAL, 1995). 3 

Typology of Self-Reported Descriptions of Degrees of Collaboration and 
Authenticity in Adult Literacy Classes 

As preparation for this study, we undertook a survey of adult literacy program 
directors across the United States (Purcell-Gates, Degener, & Jacobson, 1998). 

Using a short questionnaire, delivered over listservs and by mail for randomly 
selected programs, we asked spokespersons of adult literacy programs to describe 
their programs. The questionnaire was structured to gain information about the two 
dimensions of interest: (a) the degree of collaboration, or power sharing, between the 
teacher and the students; and (b) the degree to which the activities and materials in 
the classes reflect real-life literacy uses. We received a total of 271 responses, 
representing programs across all regions of the U.S. Results showed a clear 
clustering of classes using materials and activities which are not authentic 
(“somewhat school-only” = 34%; “highly school-only” = 41%; Total = 75%) and 
teacher-directed rather than collaborative (“somewhat teacher-directed” = 61%; 
“highly teacher-directed” = 28%; Total = 89%). A full 73% of the reporting 



" For this study, we defined collaborative practice in terms of power sharing between teacher and student 
without, necessarily, the libera! orv purpose inherent in Freirean-based definitions of dialogic practice 
(see Leistyna, Woodrum, & Sherblom, 1996). 
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practitioners described adult literacy classes that are teacher directed and involve 
their students primarily with activities and texts that are designed for only school- 
type settings and not for use in the out-of-school lives of the students. These 
categories are elaborated upon in Chapter 5. 

This report documented that adult literacy classes, at least as described by 
those involved in the running and teaching of them, dramatically fail to reflect the 
prevailing beliefs about “best practice” for adults just summarized. The authors (in 
review) conclude: 

This scenario is almost the direct opposite of that being advocated by 
the great majority of adult literacy academics and policy leaders in the 
United States. Thus, this result presents the field with an interesting 
conundrum: why is there such a gap between public policy and theory 
and actual practice, at least as described by practitioners? This is a 
serious and significant question to be addressed by the field of adult 
literacy, a field that has received increased portions of public funding 
and public attention in the last 10 years (p.22). 

In this report, the authors clarify that their intention was not to test the 
efficacy of instruction that is both authentic and collaborative, but rather to lay the 
groundwork for such inquiry. The present study takes that descriptive study to the 
next step and begins to explore outcomes of adult literacy instruction that can be 
described along these two dimensions. 

Research Questions 

In summary, there is only some data and much theory about the “best” kinds 
of instruction for adult literacy development. There are good reasons to use change 
in literacy practices as an outcome of adult literacy instruction, given what we know 
about the need for practice of newly learned skills and about the emergent literacy 
benefits of many and varied uses of print in the lives of young children. No one has 
yet, however, systematically explored the relationships between types of literacy 
instruction and change in literacy practices on the part of adult students. The 
research questions for this study, thus, were: What are the relationships among (a) 
the degree to which adult literacy classes employ real-life literacy activities and 
materials; (b) the degree to which students and teachers share decision making; and 
(c) changes in students’ out-of-school literacy practices. The outcome measure was 
change in out-of-school literacy practices of the students, both in frequency per type 
of practice and in types of practices. 
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CHAPTER 3: PARTICIPANTS 
Recruitment 

Teachers/classes and students were enlisted in this study through a process of 
snowball sampling. We initially contacted all respondents to the just-completed 
typology study, described above. Simultaneously, we put out calls over adult literacy 
listservs, through the National Center for the Study of Adult Learning and Literacy 
(NCSALL) contacts, databases, and publications. In addition, we began personally 
contacting state and regional adult education programs, asking for volunteers. To 
participate in the study, the site needed the following (a) at least one teacher willing 
to participate; (b) at least three students from that teacher’s class willing to 
participate; (c) an identified local data collector, willing to be trained by us and to 
collect the data over the course of the study. 

Criteria for Inclusion 

In recruiting students, we sought to target adults who were working to 
improve their literacy skills. We considered students who were in the basic 
education levels, ranging from acquisition of basic literacy to acquisition of literacy 
skills needed to pass the GED. We established two sets of criteria for a cut off level 
for the ESOL students and the English-only students. ESOL students were only 
accepted into the participant pool if they did not possess high school diplomas or 
equivalents from their native countries. Further, they were not accepted if based on 
self-report they practiced high levels of literacy in their native language. While many 
of these students were in ESOL classes, for our purposes, they needed to be working 
on their literacy skills as well as on learning to read, speak and understand English. 
For English-only students, possession of a high school diploma or its equivalent did 
not disqualify them from the study, but they needed to be in their adult class to 
improve their literacy skills. Participants were paid $10 for each questionnaire 
interview. 



Portraits of Participants 

The results of the analysis are based on the responses of 159 adult literacy 
students. Their ages ranged from 1 8 to 68. They attended adult literacy classes in 22 
states, fairly evenly distributed across the mainland United States. The classes they 
* attended represented the range of adult education class types including Adult Basic 
Education (ABE), English for Speakers of Other Languages (ESOL), classes 
specifically focused on helping adults gain their General Educational Development 
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(GED)credential, and different versions of family literacy classes. Their literacy 
levels upon entry to the classes they were currently attending ranged from preliterate 
to 1 1+ grade levels. Descriptive statistics for the participants along these dimensions 
are presented in Chapter 5: Analysis. 

Personal Histories 



The participants in this study report varied histories, many of them reflecting 
backgrounds of poverty, hardship, and schooling difficulties, all of which will seem 
representative and familiar to those educators who have worked with adult literacy 
students. One man, a father of two, in an ABE class related that he found himself 
living on the streets at a fairly young age: 

1 got my own place; then 1 had an injury. I had a lot of trauma. . .it was 
getting hit by a vehicle. 1 ended back in the hospital and then from 
there a group home. . .for 3 1/2 years. Now I’m in my own apartment 
finally after all that. I was like 27 when I first got my apartment after 
all of this havoc. 

Another woman, working with a tutor, wove memories of her learning difficulties 
throughout her stories of personal struggle: 

It was about in the 3 rd grade when I realized that I was not able to read 
too well. I was 1 5 when my brother was shot and killed. . .1 was never 
actually a teenager since that. I washed dishes in my mother’s and 
daddy’s cafe and then I married and then I became a mother at 17. 

When I was 21,1 divorced him and married my husband that died in 

’95. When I was in the 3 rd grade, I had a teacher named Mrs. K , 

and she wanted to spend the summer with me to, you know, teach me 
how to read. Well, somethin’ had come up and my aunt wanted to go 
visit Grandma and Grandpa in California, so my uncle, he told my 
aunt to take me. So I went to California and the day after I left, my 
teacher come to pick me up. So that was the end of that. I went to 
special ed, and I thought that I was doin’ pretty good. They seen an 
improvement. So they took me outa there and put me in the 5 th grade. 

Well, that threw a shock on me. Because I wasn’t wantin’ to leave. I 
wanted to stay there because I was doin’ so good. I went to the 
bottom of the class again, and, right then, it just seemed like it was 
more down hill. I don’t know why but my mind just closed. Up ‘til 
the 8 th grade. And that’s when they kicked me out. 
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After her children left home, she attended a literacy program for two years. 

However, 

It was after that that my son died and I lost my memory. I had had it 
lost for five years, after he died. My daughter also died. My husband 
died in ’95. So, I found my boyfriend and I found a literacy school 
and I stayed. My daughter got me into it ‘cause she was goin’ to try to 
get her GED. 

Immigrant ESOL students often related stories tied to poverty, exclusion, and 
violence. One woman from Guatemala reported: 

I was in school in Guatemala. I have to walk 3 hours. I have to get up 
by 5 o’clock in the morning. I don’t remember if I eat breakfast or 
not, but I have to get up early to walk to the school, and the school 
start by 8 o’clock (and) it’s a long walk. I was 10 when I started 
school. (I went) only 3 years. Because there was no school closer to 
where my family lived, so I can’t go anymore. (It was) dangerous. In 
1993 I was in 3 rd grade. So at that time there was people getting killed 
and everything. And the guerrillas and the government were fighting, 
so there was no way I could go to school. There were no kids going to 
school at that time. The teachers, they returned to the city. 

Reported Personal Reasons for Failure 

Many of the participants also reported internal characteristics that accounted 
for their failure to learn to read and write as children. In fact, for the most part these 
adult students had shouldered the blame for their previous difficulties, a phenomenon 
reported elsewhere about ABE students (Purcell-Gates, 1995; Purcell-Gates & 
Waterman, 2000). This is captured by . . . 

I was a weird person back then. I’m serious; I couldn’t, like I said, I 
couldn’t remember nothing. You know my mom would send me to 
the grocery store to get a gallon of milk and some bread. I would 
walk out the door, and I had to turn back ‘cause I didn’t remember 
what she said. . . .and I guess hanging around with the wrong crowd 
didn’t help either. 
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I had no confidence... I had my goals too low and I just had to realize 
that I need to take care of my needs. 

...it’s really hard for me. ...with words that sound the same and 
they’re spelled differently. Like “there,” “they’re,” and “their.” That 
was hard for me ‘cause I was always usin’ the wrong “there,” you 
know (chuckle). 

I knew I had a lotta good tales I could tell. But I just couldn’t get it all 
down, you know, ‘cause it’s still difficult for me to talk. ‘Cause I 
have a speech impediment. And I thought it would help writin’it 
down stead of tryin’to talk. Every time I opened my mouth, they was 
kids all around makin’ fun of me. So it was hard. But if I’d of 
realized then that every child had their own little piece of somethin’ 
that was wrong. I thought I was the only one. 

It don’t stick in my brain. It’s back there, I just have to open it up. 

I couldn’t keep it in my head. That’s what was goin’ on with me. I 
don’t like to read. I mean I am lazy at it anyway. 

I walked through the doors here in with a third 

grade level reading. Pronouncing out words is still awful hard for me 
- even hearing out the sounds, that’s a struggle. 

When I started school here I didn’t hardly know how to read and write 
and nothing. There were words that I should have been able to spell 
and all, I wasn’t able to spell ‘em. Things that I should have been 
able to understand, I didn’t understand them. 

Extrinsic Attributions for Failure 



Often, though, the same students would offer up other explanations or 
recounts for the barriers they saw as standing in their way to full literacy. The 
following are examples of these, in addition to stories of violence and lack of access 
to schooling reported by many immigrants: 

I was in a special education schooling system out in and 

they didn’t even want to teach me. I was treated really badly at 
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school. The kids made fun at me. They kept saying stuff that beat me 
down. 

I was in the doctor’s office here in and they found out 

that in my right ear I have 75% no hearing in it. And my left ear is 
perfect. I have problems with my ‘1’ sounds, and when I do hear I 
have to look at people. They don’t know if it’s due because I had a 
manic (sic) fever when I was a child, or because of my mechanical 
micro (sic) valve, and I’m not getting enough blood or oxygen to my 
brain and hearing area. 

My parents had died and left me, the oldest boy, to take care of the 
rest of the older ones, so I didn’t have a chance to go to school to 
learn to read and write because I had to start working when I was very 
young. I was about 1 1 years old when I started on a public job. 

My family broke up when I was very young. We were in foster homes 

for a while, and then from there we went to 

Orphanage in . I tried going back to school. I tried 

going to , but I felt like I was being shunned from 

the other kids. ..Sol just told my mother I wasn’t going anymore. I 
was going to go and work and help her out. 

I was classified as MR. That stands for “mentally retarded” and things 
like that. I was told that I would have to have someone watch over me 
the whole time, and you know being in the room with people talking 
not with you. You’re just there. They’re talking to someone else, so 
you’re just another body, and they think so incapable, and I was at the 
point where I felt that I was so incapable that I better have everyone 
talk to someone for me this and that and stuff. I had no confidence. 

Sources of Support and Motivation 

Despite such difficulties, the participants in this study were all fully engaged 
in continuing their education, and a common theme of support emerged from 
interviews with them. This support came from different places, people, and sources 
but it all was important and key to the motivation driving these students to attend 
class and work on their literacy skills. Often, teachers were cited as providing 
important support and encouragement: 
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There’s sometimes things are a little bit too personal; you don’t wanna 

talk about it, but I would talk to (teacher) about it, you 

know, and it feels good; it feels good to let it out. It feels good 
because you feel like you lost 10 pounds, you know. 

There was no boring time. Sometime I’m really tired from work or 
something like that, and talking to her (teacher), she always give me 
something to make me feel better, taught me something. . .if I just 
really felt lousy going to school, I’ll talk to her. She’ll ask how’s 

and I tell her what wrong with and things like that 

and then she will talk to me and after that I feel okay and do my 
homework. 

I have my reading tutor for a year and a half now. She’s been very, 
very patient with me. She’s been helping me build up my sounds and 
show me how to do (math). We learn things together. . . I’ll do just 
anything for the Adult Learning Center for their helping me out and 
being very patient with me. 

. . .the woman that teaches us helps us quite a lot. Any problem that 
we might have, she tries to help. 

We got a lot of teachers in here and you can go to them and ask them 
different things and they’ll try to help you, or show you. They are 
very lovely people. 

Participants also reported receiving significant support from family members. 
At times this support was given to encourage the participant to continue schooling. 
At other times, family members provided the support needed to get by in life with 
low literacy skills. Occasionally, participants told us about family members’ 
attempts to teach them to read and/or write: 

(My husband) took care of everything. He tried to teach me, but he 
went at it the way that I didn’t think it shoulda been. I’d be a’writin’ a 
story or tryin’ to spell something and I’d ask him, and he would go 
into what other meanings the word has. But I was wantin’ to know 
how to spell it. And I’d swear up and down I wouldn’t try again, but 
then a month or so I’d bug him again. 
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(My husband) he’d come in and he’d set down and, you know, he’d 
help me read and he helped me a whole lot with the writin’. But, he 
started to work so that cut that down. But before long, it’d be winter 
(when he doesn’t work as much) and we’ll be able to stay here, and 
when you’re together 24 hours a day, you might can do a little bit. 

My husband, he always read. He read to me and the kids. 

Before, I couldn’t go out alone. I had to go with my husband so he 
could help me. He used to do all, everything. 

He (first husband) took are of everything. I didn’t need to (read and 
write), no. 

Yeah, they (her parents before she got married) took care of 
everything (reading and writing tasks). 

My mother, when they run the cafe, I waited tables and I didn’t have 
to know how to write to do that ‘cause I could go tell my daddy, and 
he had a memory, you know, like a horse. You tell him, and he’d 
keep it all in order. 

My wife, she used to help me do everything. She used to have to 
make up all the (church) program, do all of the reading and writing 
going on. 

Support and encouragement was also offered by people other than family 
members, according to the participants. 

Reading, like, all these letters that I’ve been getting for child support. 

I used to have people read ‘em for me. Even friends would write to 
me, and I would have people read ‘em for me. 

Sometimes I can’t understand what the words are. . .but at least I try to 
read and somebody, people especially in the store, (will help). 

(My caseworker) she sent me in. She just said, “Okay, now, go to 
school.” 
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Well, this other girl was going to join it (adult literacy program), and 
she didn’t really want to be alone in the class, so she asked me, why 
don’t I join it? I says, “sounds interesting,” so I tried it, and I’ve been 
there ever since. 

Finally, several participants shared their belief that God had supported 
their efforts: 

. . .but I know now that the Lord was dealing with me in studying and 
then learning about his word and everything, so I was able to learn 
more about how to spell different things in the Bible than I did in 
history. 

Actually, God is first, you know, and I study going to church and talk 
to God. I talk to God a lot, and He helped me out a lot. He’s pushed 
me a lot, you know. 

I guess God chooses that day for you, you know, and I guess this is 
my year. 

While the participants were clear that many people helped and were helping 
them in their struggle to learn to read and write, they also talked much of their 
intrinsic motivations which ranged from personal feelings of empowerment to the 
desire to be better workers or better parents. Following are selected quotes reflecting 
these feelings: 

It feels good (to be learning to read). I wish I had just had all this 
power back then. But it’s never too late. 

(I’ve felt I could learn to read) ever since my little one has been in my 
life. If I would’ve known he was gonna change my life for the 
better... 

I’m testing myself. I wanna know what I’ve really learned. I really 
wanna know how far I’ve gone because if I am reading a little bit 
more, I’ll probably learn more like science, or whatever - things I 
didn’t know back then, I’m learning now. 

I have the baby now, and I read books with him. He’s got his own 
little library. 




17 



NCSALL Reports #17 



November 2000 



Hopefully, I’ll get my license (GED) and get more respect. You 
know, people see that license, you get more respect. 

I wanna be first! I wanna do everything first now. I wanna make up 
for all the years. I’m serious, for all the thirty nine years. 

Well, what really interested me was I was wantin’ to write a story 
about my brother’s death. About how he was killed and how much 
time did the guy that killed him spend in jail. The boy that killed my 
brother, when he got out, he was out a year and then he killed a boy. 
And I was wantin’ to investigate that ‘cause I don’t know if he was 
shot, when the police shot him, but I never did find out if he was 
killed. 

I just wanted to sit and read a book. I’d see people sittin’ back, 
re'laxin’, readin’ a book. And I’ve always wanted to do that. 

What made me decide to go to school? It was because after I wasn’t 
working or nothing like that I needed something to put my time into. 

It’s a great encouragement for your kids, whether they are little to big, 
or big to little, you have a positive role model to them and that’s what 
counts. 

We wrote an essay about why we wanted to get our GED, and I said in 
mine that I want to earn mine; I don’t want people to give it to me. 



Summary 

The adult literacy students who participated in this study appeared to represent 
the range of students who are working to improve their reading and writing abilities in 
the various iterations of adult literacy classes in the United States today. They were both 
native-born and foreign-bom. They ranged in age from 1 8 to 68. They had all failed to 
become fully literate as children for a variety of reasons, and they were currently 
learning in classes or in tutorial arrangements that reflected a range of configurations: 
ABE, GED, Family Literacy, Evenstart, and ESOL. Their personal histories reflected 
the economic and social struggles typical of poor and marginalized groups. Throughout 
their lives they had learned coping strategies to make up for their literacy difficulties, 
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leaning on others as well as finding their own inner strengths. At some point, for 
different and varied reasons, each had decided to make another commitment to learning 
and each was succeeding to differing degrees. Since adult students do not persist in 
programs where they feel unsuccessful, and we only included those students who were 
currently attending classes, it is fair to say that all of the participants in this study were 
experiencing academic success at the time we were involved with them. 
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CHAPTER FOUR: METHODOLOGY 
Design 

This study was descriptive and correlational in design. Multiple methods 
were employed to describe adult literacy classes along the two dimensions described 
above as well as to document (a) the full-range of literacy practices engaged in by the 
adult learner participants and (b) the changes, as self-reported, in literacy practices by 
the adult learner participants. For purposes of this study, change in literacy practices 
was operationalized to mean both the adoption of new types of literacy practices and 
increases in the frequencies with which familiar literacy practices are engaged. The 
relationships between the types of instruction experienced by the participants and any 
change in their literacy practices were captured through analysis of correlations 



Data Collection 



Classroom Data Protocols . 

To type the classes along the two dimensions of interest, we developed three 
different protocols to allow us to triangulate our data. First, a five-page teacher 
questionnaire was developed which incorporated short-answer questions, check-off 
items, and Likert scales. The items elicited information about specific literacy 
activities and their purposes; materials and texts read and written; the involvement of 
students in the decisions to use these materials; the extent to which teachers felt the 
activities and materials reflected real-life activities and materials and the extent to 
which the students collaborated in choosing them; how student work was assessed 
and the extent to which students were involved in assessment; and the extent and 
type of student involvement in the overall program and program administration and 
policies. 

Secondly, a protocol was developed for the data collector to use to describe 
the instruction along the two dimensions after observing a class in session. This 
protocol included holistic descriptions of the class sessions as well as individual 
items, including Likert scales, similar to those on the teacher questionnaire having to 
do with activities and materials used. Data collectors were instructed to first 
familiarize themselves with the content of the protocol and to take only field notes 
during the actual class observations. The protocol was to be filled in by the data 
collector immediately following the class observation. 
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Finally, a protocol was developed for the data collector to use in an interview 
with volunteer students, without the teacher present. This interview asked the 
students to describe the types of things they read and wrote in class and to rate the 
degree to which (a) they thought these activities and texts were “real-life” and (b) the 
degree to which they, the students, participated in choosing the activities and texts. 
The descriptions of class activities and texts were collected from the group 
discussion, with all contributions accepted. Responses to the Likert-scale items were 
recorded via a “vote” procedure, where students were to individually vote for the 
response they felt most accurately captured their perceptions. The data collector 
filled in student responses to the questions on the protocol at the time of the 
interview. 

With the use of the Teacher Questionnaire, the Class Observation Protocol, 
and the Student Interview Protocol, we collected information about the two 
instructional dimensions of interest to the study — teacher/student collaboration and 
authenticity of materials and activities — from three different perspectives: the 
teacher, the students, and the data collector observer of the class. This allowed us to 
triangulate the data on which we depended to characterize the instruction associated 
with each student participant. 

We piloted each of the three protocols in adult literacy classes not 
participating in the study. Each protocol was amended and adjusted as a result of this 
pilot work until we were satisfied that potential confusions inherent in the questions 
were clarified and that each question- or item- elicited the information we sought as 
it informed our research questions. In addition, for the Classroom Observation 
Protocol, three members of the research team used it to observe three different adult 
literacy classes, filling it in as appropriate. These completed protocols, 
demonstrating different types of classrooms, were copied and included as models in 
the training material for each data collector. 

Student Home Literacy Questionnaires. 



To gather data for our dependent variable — information about change in the 
literacy practices of the adult learners — we developed an extensive questionnaire for 
use with each student. The items on the questionnaire asked about specific literacy 
practices such as reading coupons, writing personal letters, reading to a child, and so 
on. The questionnaire included 26 items pertaining to adult reading, 14 items 
pertaining to adult writing, 7 items pertaining to adult-with-child reading, and 3 
items pertaining to adult-with-child writing. 
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The items on the questionnaire came, for the most part, from a participant- 
observation study of reading and writing in the homes and lives of 20 low-income 
families (Purcell-Gates, 1996). Thus, they were derived from actual data to ensure 
grounded validity. To further ensure that we included all types of possible literacy 
practices, we piloted the questionnaire on five middle-income families. We added 
several items following this procedure. 

For each item, the student was asked (a) In the past week, did you read/write 
X? (b) Can you show me or tell me about an example? (c) (If no) Have you ever 
read/written X? (d) (If yes) When was the first time you did this sort of reading? (e) 
How often? (f) Do you still do this sort of reading? Quotes and comments were also 
captured with each item. To provide a reliability check on the item of interest to 
us — change in literacy practice — we inserted between every fifth item, a question 
that summarized the preceding five items and responses and asked specifically about 
change: “I’ve just asked you about X, X, X, X, and X. Did you just begin to read any 
of these types of print (texts) after you began attending your literacy class? Specify 
which ones. Do you think you’ve read any of these items more often since you began 
attending your literacy class? Specify which ones.” At the end of the interviews, 
data collectors asked if the participants could think of anything else they had read or 
written during the last week or ever. They were then invited to walk through the 
house to discover or point out texts they had read or written. 

The Student Home Literacy Questionnaire was piloted with 45 adult literacy 
students who were not participating in the larger study. During the piloting, we 
refined items, questions regarding demographic and personal information about the 
participants, and directions to the data collectors within the questionnaire. For 
participants who spoke mainly Spanish, we used a Spanish version of the 
questionnaire, translated by a native- Spanish-speaking member of the research team. 

Spanish-speaking data collectors collected data from primarily Spanish-speaking 
participants. 

While we had students in the participant pool who spoke non-English 
languages other than Spanish, we provided data collectors only for the Spanish- 
speaking students. This was due to the difficulty of finding, and employing, data 
collectors who spoke other languages. Therefore, for the non-native English 
speakers, other than the Spanish-speakers, a requirement for their inclusion in the 
study was that they speak and understand English well enough to answer the 
questionnaire items in English. We also had native Spanish-speakers who 
participated completely in English. They were judged by the teacher and themselves 
to be fluent enough to do so. 
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Since there is no theoretical or empirical reason to suggest that the language 
in which the reading and writing occurs is related to the degree to which young 
children in the home learn about emergent literacy concepts, our work with ESOL 
students focused on changes regarding specific text types, rather than the English 
skills they may be acquiring and using. To ensure that newly acquired English 
language skills were not conflated with changes in literacy practice, a separate form 
of the Home Literacy Questionnaire was developed for all ESOL students. On this 
form, each literacy practice question was asked as it applied to engagement in that 
practice for each language spoken by the student. This allowed us to control for 
reported changes in literacy practice that represented a shift into a new language. 

Procedures 

Site Management: Data Collector Training 

Each of the four members of the research team took responsibility for 
overseeing the data collector training, collection of the data from the site data 
collector, and coding of the data from an assigned group of sites. Data collector 
training involved the research team member flying or driving to the adult literacy 
class site and meeting for approximately 4 hours with the data collector. At this 
meeting, the data collector was given two accordion folders, one of which contained 
the following: (a) protection-of-human-subjects information letters and agreement- 
to-participate-in-the-study forms (to be returned to the research office); (b) protocols 
for the class data; (c) copies of the Student Home Literacy Questionnaire; (d) 
stamped priority mail envelopes to mail the data to the university research base; (e) 
printed instructions for conducting the data collection and using each of the 
protocols; and (f) expense sheets and instructions on submitting expenses. The data 
collectors were instructed to copy all data before mailing it and keep the copies in the 
second accordion folder. At times, the research- team member met with teachers and 
program administrators to discuss the project. 

Site Management: Site Maintenance 

A major challenge for this study with its many far-flung sites was 
maintenance of a site. Following the data collector training, the individual research 
members maintained regular contact with their data collectors by phone and/or email. 
The purpose of this regular contact was to help problem-solve issues of data 
collection as they arose, locate new classes or students as needed, communicate with 
teachers and program administrators as needed, and so on. 
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Two members of the research team served as data collectors as well as site 
managers for sites close enough to the university to be visited by car on a regular 
basis. A third member of the research team managed sites around the Chicago, IL 
area since she was living there at the time. She also served as data collector for 
several of these sites. One member of the team was a Spanish national and she 
managed most of the ESOL sites and translated all of the protocols into Spanish. She 
also transcribed all of the audiotapes made in Spanish into English. 

As mentioned above, we struggled over the course of the study with problems 
emanating from lack of proximity to our research sites. Often trained data collectors 
would “disappear” (at times, never to be heard from again) or programs would close 
before data could be collected. One program in New York City actually closed and 
reopened under new directorships three different times before a data collector could 
ever gain access to the site. With each change, we had to elicit new teacher and 
student agreements to participate. We also lost two different data collectors trained 
for this site as time passed and other opportunities arose for them. Related to this 
proximity issue was an ongoing problem of collecting data from the data collectors. 
Site management, in this sense, meant constant calls trying to locate data collectors 
to ascertain why promised data had not reached the research office. 

Another issue that plagued us to the end was the administration of the 
questionnaires to second-language participants. Many of the data collectors failed to 
obtain information about literacy practices in both the native language and English. 
Also, some ESOL participants did not wish to give us information about their literacy 
practices in their native language. This made it problematic forjudging whether or 
not a practice began or increased in frequency of use after the participant began the 
class. As a result, we had to drop a number of our ESOL participants from the final 
data pool. 

Site Management: Central Data Base 

A Project Manager, based entirely at the university office, was hired to 
maintain the central database as data were received, coded, and analyzed. The 
Project Manager maintained all participant records, permission forms, and codes. 

She also maintained contact information for programs, teachers, and data collectors. 
In addition, she recorded the minutes of all research meetings, communicated with 
data collectors, teachers, and program administrators as needed, and served as a 
liaison among research team members, data collectors, research center officials, and 
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the finance office of the university. Toward the end of the analysis period, this 
person also helped with qualitative data coding and entry. 

Site Manaeement: Research Meetings 

One of the challenges of a multi-site research study is to maintain consistency 
of data collection across the sites. To this end, two-hour research team meetings 
were held weekly for the length of the data collection — 20 months. At these 
meetings, solicitation of research sites was discussed and developed, assignment of 
sites and plans for data collector trainings were made, issues around the data 
collection and delivery were discussed and decisions made, and procedures for 
coding and analyzing the data were developed. 

Following the data collection period, research meetings were held 
approximately every 2 months to discuss issues of data analysis and to plan for the 
collection of final class and student-level information dictated by the statistical 
analysis (see Analysis section for more detail). During the analysis period, much of 
the communication among research team members was carried out via email as the 
Principal Investigator had moved to a university in Michigan and, as stated before, 
one of the research team members lived in the Chicago area. The bi-monthly 
research meetings were held either at the Harvard research office or at professional 
conferences attended by all team members. On several occasions the Principal 
Investigator met with the Chicago-based researcher team member in Chicago during 
the reliability check period. 

Class Data Collection 



To collect the class data needed to type the individual classes along the two 
instructional dimensions of interest, the data collector first delivered the 
questionnaire to the teacher and arranged a time to observe a “typical” class, after 
which he/she would collect the questionnaire. During the class observation, the data 
collector assumed an unobtrusive position in the class and made notes on all that 
occurred as well as physical descriptions of the classroom, equipment, shelves, where 
students sat, and so on. During this visit, either during a break or right after the class, 
the data collector gathered a group of volunteer students to conduct the student 
interview. Following the observation, the data collector used the field notes to fill in 
the protocol for the observation. The data collector also read over the completed 
teacher questionnaire and arranged for a short interview with the teacher to clarity 
any ambiguous answers and to ask questions which may have arisen during the class 
observation. 
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Literacy Practices Questionnaire Data Collection 

Data collectors conducted the literacy practices interviews in the homes of the 
participants. The context of the home was intentionally chosen to (a) remove the 
students from their literacy classes to ameliorate the tendency of participants in self- 
report studies to give the interviewer what they think is expected (in this case 
information that the students may think will cast their classes in a positive light); (b) 
to help the students think about real-life literacy practices -- rather than school-based 
ones — by placing them in the context in which many of those practices would have 
occurred and from which they could select examples. The interviews were repeated 
every three months for as long as the student was in the program up to four times or 
until the end of the data collection period, which stretched over 1 8 months (although 
students began with the study at different points, reflecting the snowball sampling 
nature of the study). Each interview/questionnaire took an average of one and a half 
hours. 



If a student changed classes during the course of the study, the data collector 
collected class data (with the same procedures and three protocols) on the new class 
and the student/class pair was treated as a new “site”. This procedure was in 
response to the fact that the individual class was the unit of interest for this study, not 
an overall program. The questions on the literacy practices questionnaire were then 
asked in reference to the new class, the class currently being attended by the student. 

Taped Interviews 

To capture more directly the voices and stories of the students, in relation to 
our research questions, we sent each data collector a tape and sample interview 
questions and asked them to select one participant who they thought would make a 
good informant and to conduct a taped interview. The interview was to focus on the 
literacy histories of the participants and the life events that contextualized their 
seeking out literacy instruction as adults, their descriptions and feelings about their 
literacy classes, and the ways in which, if at all, their lives, including their literacy 
practices, had been affected by their instruction. 

A total of 40 tapes were sent, and 1 7 were returned. Three of these were not 
used. Two were inaudible due to mechanical malfunctioning of the tape recorder 
used, and one was of a student who had been dropped from the participant pool by 
the time we received the tape due to an inappropriately high level of literacy ability. 
These audiotapes were transcribed and coded for themes in the responses. This data 
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is used only for illustrative purposes in this report and not as data for the analysis 
conducted to answer our research questions. 

Other Relevant Student Data 



Following the literacy practices data collection, all of the teachers were 
contacted and asked about the students’ incoming literacy levels, attendance patterns, 
and number of hours per week the class was held. For literacy level, we sent the 
following request: 

At what level of reading and writing would you say the student was 
when s/he joined this class? This is a subjective question and can be 
answered in several ways. If you can give a specific grade level (i.e. 
reading at a 5 th grade level), that would be great. Otherwise, more 
general terms like low-literate or non-reader would also be helpful. 

Please indicate what evidence your response is based on (e.g., formal 
assessment, teacher observation, prior reports, etc.). Note: If the 
student’s native language is other than English, please tell us what 
level the student was in both first language and English. 

Teachers assessed their students differently. Some gave a grade level, some used 
terms like “pre-literate, ’’others used TABE or CASAS test scores. As a research 
team, we created a five-point scale for literacy level, and together went over all the 
different teacher reports of student level and determined where on the five-point 
scale students fell. Students received a 1 if they were considered preliterate; 2 if their 
reading was at the grade 1-3 level; 3 for grade levels 4-7; 4 for grade levels 8-10; and 
5 for grade levels 1 1 and higher. 

For attendance, the teachers were given a five-point scale for describing their 
students’ overall attendance patterns: 1 = Rarely Attends; 2 = Sometimes Attends; 3 
= Attends More Often Than Not; 4 = Often Attends; 5 = Attends Very Regularly. 

All of this information was used in the final analysis. Tables 5 and 6, to be found in 
Chapter 5: Analysis, display this information. 
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CHAPTER FIVE: ANALYSIS 
Class Data 

Data on the individual classes were coded and triangulated for typing along 
the two dimensions in the following way: Information from each of the three 
protocols was transferred to one coding sheet into a matrix with the following 
headings: (a) Source (Class Observation, Teacher Questionnaire, Student Interview); 
(b) Activity/Material (e.g., “writing correspondence letters,” “filling in worksheets,” 
“reading children’s books”); (c) Purpose (e.g., “to thank guest speaker,” “to practice 
grammar,” “to encourage parents to read to their children”); (d) and Assigned 
(choices were “by the teacher,” “by the teacher and student together,” and “chosen by 
the student”). All Likert scale numbers from each protocol were transferred to the 
coding sheet as well as responses to short-answer items. The coder then holistically 
assigned an authentic/school-only “score” as well as a collaborative/teacher-directed 
“score” to the class and placed the intersection of these onto a miniature dimensional 
grid located at the bottom of each coding sheet. This was later transferred to the 
central database. Criteria for typing the classes are contained in Tables 1 and 2. 
Coders were advised to consider proportion of instructional time devoted to relevant 
types of activities/texts whenever possible when making decisions about assignment 
of class type. 
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Table 1 

Criteria for Classifying Classes Along the Authentic/School-Onlv Dimension 



Type 



Criteria 



Highly 

Authentic 



Somewhat 

Authentic 



Somewhat 

School-Only 



Highly 

School-Only 



Classes that use only realia, or texts that occur naturally in the 
lives of people outside of a classroom (e.g. newspapers, 
journals, novels, work manuals, driver’s license materials 
etc.). Further, the reading/writing activities that occur with 
these texts are the same as those which would occur with the 
texts outside of a learning situation. It is the confluence of 
these two factors that make programs highly authentic. 

Classes in this category may mention skills and may use some 
published textbooks and workbooks, but student work is 
heavily concentrated on real-life texts and issues. Classes in 
this category may use real texts with activities that mimic but 
are not the same as those which would occur with the text 
outside of a learning situation. For example, letters-to-the 
editor are written but not be sent. 

Classes are more highly focused on skills, with the majority of 
activities focused on phonics work, grammar work, 
workbooks, etc. Materials tend to be published textbooks and 
workbooks, though some mention may be made of authentic 
materials or activities such as Language Experience Approach, 
newspapers, journals, etc. This type of class tends to use 
facsimiles of real texts for learning to read these types of texts, 
e.g. photocopies of newspaper articles that are not current and 
not necessarily about topics of interest to students. 

Classes have a set curriculum with a focus on skills, 
phonics, flashcards, etc. Most, if not all, materials are from 
publishers, and there is almost no mention of authentic 
materials or activities. These classes most likely do not use 
real life texts in any form. If they do, they are made up for 
teaching purposes and used for only skill learning (i.e. not for 
communicative purposes). 
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Table 2 

Criteria for Typing Classes Alone the Collaborative/Teacher-Directed Dimension 



Type 


Criteria 






Highly 

Collaborative 


Classes where students work with teachers to create the 
course, choose the materials, activities, etc. Students are also 
involved in all aspects of the program, may serve on the board, 
make decisions regarding meeting times, class rules, class 
structure, and location, etc. Students may also work to publish 
newsletters and may help in recruiting new students. 


Somewhat 

Collaborative 


Classes where student input is obviously influential on the 
activities of the class. These classes reflect an ongoing 
flexibility re curriculum to allow for response to students’ 
interests and/or expressed needs. There is collaboration in 
choosing course content and activities. Students are in charge 
of their own learning. 


Somewhat 

Teacher-Directed 


Classes where students’ goals, interests, and/or needs are taken 
into account when creating course content. Students have 
some input into class content, usually in the form of interest 
inventories, students’ goals or IEPs. Teachers encourage 
student input. Students typically choose from materials and 
activities that have already been selected by the teacher. 

Classes in this category consider themselves to be client 
driven though ultimate course decisions typically rest with the 
teachers. Teachers give needs assessments throughout the 
students’ time in the class. Teachers and students periodically 
reflect on goals and whether or not the class is meeting them. 


Highly 


Classes where students have little or no input into 
course 


Teacher-Directed 


content, activities or materials. Students may be given a needs 
analysis when they start the program but needs are not 
continually reevaluated. Teachers may say that demographics 
of the students impact course content. Often a set curriculum 
is followed in these classes. 
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Interrater reliability of the class typing was achieved and assessed in the 
following ways. First, to achieve initial agreement on class typing, the entire 
research team worked together to type approximately five classes. During these 
sessions, we talked aloud about the criteria for each decision and came to agreement. 
After these sessions, each member of the team coded their classes from their 
collection of sites for which each was responsible. However, data collected by a 
research team member, as opposed to a data collector not on the team, was coded by 
another team member to ensure comparability of coding decisions across all sites. 

Secondly, an interrater reliability check was conducted by randomly selecting 
one-third of the coded sites and recoding them. The recoding was done 
independently by the three research team members who had not coded the site 
originally, with the exception of those sites where the data collector was a team 
member. These were distributed to the two members of the team who had not 
originally coded the site nor had collected the data. Pairwise correlations of the 
results (Spearman’s Rank Correlations) ranged from r = .56 to r = .76 for the 
Collaborative/Teacher-Directed dimension and r = .60 to r = .76 for the 
Authentic/School-Only dimension. Considering these interrater reliabilities too low, 
we initiated a third procedure. 

The two team members with the highest overall agreement in the initial 
reliability check met and clarified criteria for typing the classes along the dimensions. 
With rewritten rubrics, they then each recoded one-half of the entire class data set, 
again assigning sites for recoding to the team member who did not collect the data. 
An interrater reliability check was run on the entire recoded class data set with the 
following results: Authentic/School-Only dimension — r = .86; 
Collaborative/Teacher-Directed dimension — r = .78. 
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Home Literacy Practices Questionnaire 

Data from the Home Literacy Practices Questionnaire that were directly 
relevant to the research questions were transferred onto a coding sheet with the 
following matrix headings for each literacy practice asked about: “Last Week;” 
“Ever;” “Since Class;” “More Since Class.” The coder marked a ‘0’ for “no” and a 
‘ 1 ’ for “yes” in each of these columns. A star (*) was used to indicated missing data 
and a ‘2’ to indicate that the question was not applicable for that participant, given 
who they were (e.g. if they did not have children to read to) or their answers to 
previous questions (e.g. if they had already said yes to reading coupons, then the 
question “Have you ever read coupons?” was not applicable). Coder decisions about 
the change columns (“Ever” and “Since Class”) were based on the questions that 
asked about dates of beginning the class, dates that practices began, and the direct 
questions about beginning the practice or engaging in it more frequently. 

Interrater reliability for the coding of the Home Literacy Practices 
Questionnaire was checked by randomly selecting 77 of the coded questionnaires 
from a total of 293 (26% of all coded questionnaires). Each of these selected 
questionnaires was recoded by another member of the research team. The completed 
code sheets were then compared for agreement of code at the individual question 
level. A total of 8,155 codes were examined. The overall rate of agreement between 
the original code and the second code was 94%. In the following categories, the rates 
of agreement of agreement were similarly high; (a) Reading in Native Language (for 
ESOL students) — 92%; (b) Reading in English (for all students) — 94%; (c) Writing 
in Native Language (for ESOL students) -- 93%; (d) Writing in English (for all 
students) — 94%; (e) Literacy Practices “This Week”— 95%; (f) Literacy Practices 
“Ever”— 96%; (g) Literacy Practices “Initiated Since Class” — 94%; (h) Literacy 
Practices “More Since Class”— 89%. 

Scaling Responses to the Questionnaire Items. 

The final purpose of this analysis was to model the changes in literacy 
practices using the theoretically-derived predictors of (a) authenticity of instructional 
activities/texts and (b) collaborative nature of teacher/student relationship. Rather 
than perform analyses on individual literacy practices, the practices were placed on a 
common scale. This was done because the reliability of responses to individual 
survey items is very low, resulting in a less powerful analysis. When aggregating 
responses over the entire set of items answered by each respondent, the reliability of 
the resulting scale is much higher (Mehrens & Lehmann, 1991), resulting in a more 
powerful analysis. 
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Item Response Theory (IRT) methodology was used to place the literacy 
practices and respondents on a common scale. All IRT estimation was performed 
using BILOG (Mislevy & Bock, 1993). IRT methodology (Lord, 1980) was chosen 
for the following reasons: 

1 . IRT methods can place literacy practices and respondents on a common scale 
even with substantial amounts of missing data. 

2. IRT methods can provide an estimate of each respondent’s engagement in 
specified literacy practices, or an estimate of each respondent’s change in 
level of engagement in specified literacy practices 

3. IRT methods can provide for each literacy practice a measure of the threshold 
of the estimate of engagement or change that each examinee needs to have a 
given probability of engaging in or changing in that literacy practice. 

4. IRT methods provide a measure of each literacy practice’s capacity to 
discriminate between respondents with high and low estimates of engagement 
in specified literacy practices or change in engagement in specified literacy 
practices. 

5. If the data fit the assumptions of the IRT model, the reliability of the resulting 
scaled scores is generally greater than that obtained through classical 
reliability analyses using Cronbach’s alpha, resulting in more powerful 
analyses using the scaled scores. 

6. Because IRT scaling tends to spread scores out, a greater variation in scores 
generally results, which in turn results in higher reliability than when using 
classical test theory methods and more powerful analyses. 

A first attempt at scaling endeavored to place each literacy practice and each 
Time One (reflecting the initial administration of the questionnaire to the participant) 
respondent on three scales corresponding to the four questions asked: (1) a baseline 
measure of engagement in literacy practice, (2) a measure of change informed by the 
development of new literacy practices, and (3) a measure of change informed by 
increased frequency of engagement in literacy practices. 

Scale 1 (literacy practice engagement scale) resulting from the IRT analysis had 
acceptable psychometric properties, in that it had a high reliability (0.87) and no 
statistically significant misfit for any literacy practices included in the scale. 

However, scales 2 and 3 resulting from the analysis (new practices and increased 
frequency of use of practices, respectively) had unacceptable psychometric properties 
in that they had statistically significant misfit for many of the literacy practices 
included in the scales. 
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There were few respondents who indicated they had begun new literacy 
practices. 4 There were also few respondents who indicated that they had increased 
their frequency of engagement in literacy practices. The rarity of respondents 
indicating change in each of the two separate measures was the apparent cause of the 
misfit of these scales. This results in little variation, and poor estimation. To address 
this problem, a more generic measure of change was created by collapsing responses 
to the questions concerning new literacy practices and increased frequency of literacy 
practices. Thus, if respondents said that they had begun the practice in question or 
had more frequently engaged in it since beginning the current adult education class, 
they were coded as having changed with respect to that literacy practice. By 
combining responses in this way, change in literacy practices became a more 
common event. 

The modified data set was rescaled, resulting in a new scale, measuring change in 
engagement in literacy practices (with change operationalized as adoption of new 
practices and/or increase in engagement in practices), with acceptable psychometric 
properties. The literacy practice engagement scale (from the first scaling attempt) 
and the change in literacy practice engagement scale (from this attempt) were applied 
to responses to the second survey, and were found to still have acceptable 
psychometric properties. 

Although these scales had acceptable psychometric properties when applied to 
second administration data, problems of inconsistency across administrations arose. 
More than one half of respondents had a lower level of engagement or change in 
literacy practice engagement at the second administration than they did at the first 
administration. This presents a problem because the level of engagement and change 
theoretically cannot drop from one administration of the survey to a later 
administration because these are measures of ever having engaged, or having 
changed at some time since the beginning of class. 

The data set was inspected for literacy practices or respondents that exhibited the 
largest amounts of inconsistency. No set of literacy practices or respondents 
exhibiting obvious inconsistencies could be identified. When inspecting the data for 
consistent responses practice by practice and respondent by respondent, 
approximately 50 percent of all possible inconsistencies were found to exist. 



4 This was not unexpected, given the full range of literacy practices asked about. As can be seen on 
the scale of engagement, most of the students had already engaged in most of the practices over their 
lifetimes. 
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We had foreseen this situation when designing the study, given the population we 
were working with. First of all, everyday literacy practices are difficult to recall for 
most people, given the ubiquitous ways in which they weave themselves throughout 
everyday life. Secondly, adult literacy students often have difficulties recalling life 
events consistently across recall sessions. Therefore, the questionnaire and its 
multiple administrations were designed to give these participants maximum 
opportunity to indicate change in literacy practices. It was believed that in the 
unlikely event that indicators of engagement and change were stable at each 
administration this data could then be used for growth modeling across time. If not, 
we could still answer our research questions. Because of the inconsistencies in the 
final data set, the multiple administrations of the survey served our original purpose - 
exploring relationships between change in literacy practices and the two instructional 
dimensions of interest. Thus, responses were collapsed again, this time across 
administrations. If respondents reported at any administration that they had engaged 
in or experienced a change in engagement in a given literacy practice, it was counted 
as engagement or change . 5 

The modified data set was rescaled, resulting in one scale measuring baseline 
literacy practice engagement and another measuring change in literacy practice 
engagement. The scale measuring change had acceptable psychometric properties, 
with an estimated reliability of 0.94, and no literacy practices exhibiting statistically 
significant misfit. 

However, the scale measuring baseline literacy practice engagement did not 
have acceptable properties— more than half of the literacy practices included in the 
scale exhibited statistically significant misfit. The reason for this misfit was 
apparently that when combining responses about literacy practice engagement across 
all four administrations of the survey, almost all respondents had at some time 
engaged in almost all literacy practices. This resulted in little variation, and poor 
estimation. 

The baseline measure of literacy practice engagement was seen as an 



5 We do not feel that the inconsistency of the responses by participants regarding change in particular 
literacy practices presents a reliability threat to the data set. On looking back over the course of the 
data collection, together with data collector comments on the questionnaires, it seems highly likely that 
on the second, third, and fourth administrations, participants interpreted the change questions (e.g., 
have you begun, or increased in frequency, to read/write XXX since you began attending your class) 
as referring to the last time they answered the questionnaire questions, not as repeat questions. 
Therefore, if they had indicated on the first administration that a particular practice was new or 
increased, they tended to answer ‘no’ to the same question about that practice on later administrations. 
This resulted in inconsistency across administrations. 
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important covariate to control for in predicting change in engagement in literacy 
practices. It was also important that the engagement scale correspond to the change 
scale, meaning that both needed to be created in the third attempt that combined 
responses across administrations of the survey. However, the poor fit of the 
engagement scale necessitated eliminating it from the analysis. Fortunately, as 
shown in the section on modeling change, the effect of losing this baseline measure 
of engagement in literacy practices was unimportant to this analysis. 

Finally, the size of the total sample changed for the final analysis. Because of 
missing data 6, it was possible to estimate a change score for only 159 of the 173 
respondents, a loss of eight percent of the sample. 

Literacy Practice Engagement Scale 

Because placing literacy practices on the continuum defined by the scale of 
Literacy Practice Engagement has theoretical importance, it is presented here. 

Because this scale was not appropriately estimated when combining responses across 
administrations of the survey, the first attempt at scaling (using only first 
administration data) is presented. 

This scaling procedure produced a single score for each respondent, 
estimating that respondent’s engagement in literacy practices, as reported by the 
participants at the time of their first interview. The estimated reliability of the scaled 
scores was 0.87, indicating that only 13 percent of the variance in scaled scores was 
attributable to measurement error. The resulting distribution of the scaled scores 
had, approximately, a mean of 0 and standard deviation of 1, as shown in Figure 2. 
The respondents on the low end of the scale reported minimal engagement in literacy 
practices, while those on the high end reported a substantial engagement. 



6 The missing data were almost entirely the result of data collectors who did not understand that non- 
native English speakers were to be asked about literacy practices in both English and their native 
languages. Thus, incomplete and unusable data were produced for a number of ESOL students. 
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Figure 2 . Distribution of Literacy Practice Engagement scaled scores. 




Propensity to engage in literacy practices 



The scaling procedure also produced three estimates for each literacy 
practice, which are displayed in Table 3. The three estimates were: (1) a measure of 
the threshold of engagement needed for a respondent to have a 50 percent chance of 
engaging in that literacy practice, (2) a measure of the literacy practice’s ability to 
discriminate between respondents with low and high estimates of engagement, and 
(3) a measure of the fit of the literacy practice to the IRT model. None of the 
individual literacy practices exhibited significant misfit, providing evidence that the 
IRT model fits the data. In addition, the literacy practices’ estimated threshold for 
engagement was spread out well along the scale, and their abilities to discriminate 
were reasonably high. 
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Table 3 

Literacy Practice Engagement Scale 



Practice discriminating threshold for Significance 



# 


ability § 


engagement t 


of misfit X 


Item description 


3* 


0.05* 


-3.00* 


♦ 


reading: labels, container print, signs 


5 


0.82 


-2.88 


1.00 


reading: calendars, tickets 


1 


1.07 


-2.52 


1.00 


reading: ads, coupons, fliers 


2 


1.16 


-2.33 


LOO 


reading: bills, bank statements, receipts 


24 


1.13 


-2.25 


1.00 


writing: names, labeling 


4 


0.66 


-2.16 


0.55 


reading: books and stories 


10 


0.82 


-2.06 


0.69 


reading: menus 


12 


1.70 


-2.03 


1.00 


reading: addresses, phone books 


11 


0.85 


-1.96 


0.62 


reading: messages, notes 


14 


0.96 


-1.90 


0.62 


reading: postal letters 


15 


0.56 


-1.85 


0.32 


reading: school communication 


8 


1.32 


-1.74 


0.64 


reading: directions (recipes, shopping lists...) 


9 


0.71 


-1.52 


0.79 


reading: essays, compositions, text for information 


16 


0.85 


-1.50 


0.43 


reading: schedules, guides 


30 


1.53 


-1.47 


1.00 


writing: messages, notes 


31 


0.89 


-1.42 


0.12 


writing: postal letters 


25 


0.98 


-1.35 


0.25 


writing: checks, money orders, gift certificates 


26 


1.21 


-1.20 


0.41 


writing: on calendars, appointment book 


29 


0.79 


-1.16 


0.19 


writing: lists 


13 


0.99 


-1.13 


0.54 


reading: periodicals (horoscope, sports...) 


28 


1.65 


-1.05 


0.48 


writing: forms, applications 


7 


1.03 


-0.94 


0.66 


reading: documents (lease, mortgage, portfolios...) 


6 


1.08 


-0.58 


0.65 


reading: comics, cartoons 


17 


0.80 


-0.24 


0.91 


reading: song lyrics 


27 


0.66 


0.04 


0.11 


writing: speeches, reflection, stories, poems 


32 


1.01 


0.05 


0.39 


writing: instructions 



§ Higher values indicate higher discrimination between respondents with differing propensities 



to engage. 

t Low values indicate low thresholds for engagement, high values indicate high thresholds. 

X The p-value of the fit statistic for each literacy practice, p < 0.05 indicates statistically 
significant misfit. 

* All respondents reported engaging in this practice. Thus, its discriminating ability is near 
zero, its misfit could not be estimated, and its threshold was set to -3.00 rather than the 
unreasonable value (-34.67) given by BILOG. 
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Interpreting the Literacy Practice Engagement Scale 

The Literacy Practice Engagement Scale in Table 3 presents a picture of how 
commonly engaged-in were the literacy practices, inquired about through our 
questionnaire, among the participants in this study as reported at each participant’s 
first interview. Those practices at the top of the scale, (e.g., "Reading labels, 
container print, signs” and “reading calendars, tickets”) were more commonly 
engaged in than those at the bottom. The level of engagement, across all of the 
participants, in the individual literacy practices is scaled here, top to bottom. 

With this scale, it was possible to give each participant a score, representing their 
literacy practice engagement. Using the metrics from the results of the scaling, a 
hypothetical participant could have received a score of -1.50 on the engagement 
scale. This would allow us to say that this person has a 50 percent probability of 
having engaged in “reading schedules and guides” (practice #16 on the questionnaire 
coding sheet) because that practice’s threshold for engagement is equal to the 
participant’s score on engagement. The lower (or higher) the threshold for 
engagement, the higher (or lower) the probability that a participant has engaged in 
that practice. For example, it is much more likely that our hypothetical participant 
has engaged in “reading calendars and tickets” (practice #5) because that practice’s 
threshold (-2.88) is lower than the respondent’s engagement score of-1.50, and the 
practice’s discriminating ability (0.82) is reasonably high. Likewise, it is much less 
likely that our hypothetical participant will have engaged in “writing instructions” 
(practice #28) because that practice’s threshold (0.05) is much higher than the 
respondent’s engagement score of-1.50, and the practice’s discriminating ability 
(1.01) is reasonably high. 

While the distribution of scores on this scale has a mean of 0 and a standard 
deviation of 1, all but two practices had thresholds of less than 0. This indicates that 
at the time of the first administration of the survey, most respondents had engaged in 
most of the literacy practices included in this survey (see Footnote #4). 

Change in Literacy Practice Scale 

The Change in Literacy Practice Scale (Table 4) presents a picture of the 
frequency of reported literacy practice change by individual practices. The scaling 
procedure also produced a single score for each respondent, estimating that 
respondent’s change in level of engagement in literacy practices. The estimated 
reliability of the scaled scores was 0.94, indicating that only 6 percent of the variance 
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in scaled scores was attributable to measurement error. Along with each 
respondent’s score, a standard error was produced, estimating the amount of 
uncertainty in the score. The scores and their accompanying standard errors were 
later used to model change in literacy practices. The resulting distribution of the 
scaled scores had, approximately, a mean of 0 and standard deviation of 1, and is 
shown in Figure 3. The respondents with low scaled scores reported minimal change 
in their literacy practices, while those with high scaled scores reported substantial 
change. 

Figure 3 . Distribution of Change in Literacy Practice Engagement scaled scores. 




Propensity to change in engagement in literacy 



The scaling procedure also produced three estimates, two estimates for each 
literacy practice as well as measure of fit, which are displayed in Table 4. None of 
the individual literacy practices exhibited significant misfit, providing evidence that 
the IRT model fits the data. The practices’ estimated threshold for change was 
spread out well along the scale, and their abilities to discriminate were reasonably 
high. 
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Table 4 

Change in Literacy Practices Scale 



Practice 

# 


discriminating 
ability * 


threshold for 
change f 


significance 
of misfit i 


Item description 


9 


0.92 


-0.29 


0.12 


reading: essays, compositions, text for information 


4 


1.15 


-0.22 


0.66 


reading: books and stories 


3 


1.90 


-0.10 


0.83 


reading: labels, container print, signs 


15 


1.02 


-0.09 


0.77 


reading: school communication 


30 


1.19 


-0.08 


0.82 


writing: messages, notes 


5 


1.67 


-0.06 


0.65 


reading: calendars, tickets 


27 


1.37 


-0.03 


1.00 


writing: speeches, reflection, stories, poems 


11 


1.34 


-0.01 


0.77 


reading: messages, notes 


13 


1.24 


0.14 


0.85 


reading: periodicals (horoscope, sports...) 


8 


1.72 


0.17 


0.86 


reading: directions (recipes, shopping lists...) 


32 


1.00 


0.18 


0.45 


writing: instructions 


1 


1.33 


0.21 


0.78 


reading: ads, coupons, fliers 


26 


1.05 


0.23 


0.86 


writing: on calendars, appointment book 


2 


0.96 


0.23 


0.11 


reading: bills, bank statements, receipts 


16 


1.11 


0.27 


0.40 


reading: schedules, guides 


12 


1.43 


0.27 


0.70 


reading: addresses, phone books 


14 


1.35 


0.29 


0.67 


reading: postal letters 


25 


0.73 


0.32 


0.62 


writing: checks, money orders, gift certificates 


29 


1.44 


0.32 


0.49 


writing: lists 


17 


1.63 


0.33 


0.40 


reading: song lyrics 


10 


1.59 


0.37 


0.48 


reading: menus 


7 


0.92 


0.41 


0.48 


reading: documents (lease, mortgage, portfolios. . .) 


31 


1.42 


0.46 


0.81 


writing: postal letters 


24 


1.45 


0.47 


0.82 


writing: names, labeling 


28 


0.79 


0.52 


0.35 


writing: forms, applications 


6 


1.05 


0.69 


0.77 


reading: comics, cartoons 



* Higher values indicate higher discrimination between respondents with differing propensities 
to change. 

t Low values indicate low thresholds for change, high values indicate high thresholds, 
i The p-value of the fit statistic for each literacy practice, p < 0.05 indicates statistically 
significant misfit. 
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Interpreting the Change in Literacy Practice Engagement Scale 

By examining the Change in Literacy Practice Scale, we can see which literacy 
practices underwent greater to fewer changes, as reported by our participants. 
Participants reported change in those practices at the top of the scale (e.g. “Reading 
essays, compositions, text for information” and “Reading books and stories”) to a 
greater degree than those at the bottom, with the frequency of change scaled from top 
to bottom. 

Each participant, as a result of the scaling procedure, received a change score that 
was then used in the analysis. For example, a hypothetical participant could have 
received a change score of 0.30 on the Change in Literacy Practice Engagement 
Scale. This person would have an approximately 50 percent probability of changing 
her level of reading postal letters (practice #31), because that practice’s threshold for 
change is nearly equal to the participant’s change score. The lower (or higher) the 
threshold for change the higher (or lower) the probability that a participant will have 
changed her level of engagement in a literacy practice. For example, it is much more 
likely that our hypothetical participant will have reported change in her engagement 
in “reading essays, compositions, and text for information” (practice #9) because that 
practice’s threshold (-0.29) is lower than her change score, and the practice’s 
discriminating ability (0.92) is reasonably high. Likewise, it is much less likely that 
our hypothetical participant will have changed in his or her engagement in “writing 
forms and applications” (practice #28) because that practice’s threshold (0.52) is 
much higher than the respondent’s propensity to change (0.30), and the practice’s 
discriminating ability (1.05) is reasonably high. 

While the distribution of scores on this scale has a mean of 0 and a standard 
deviation of 1, about two-thirds of the literacy practices had thresholds of larger than 
0. This indicates that most respondents had not reported change in most of the 
literacy practices included in the survey. This can also be taken to indicate that the 
validity threat in the tendency to give socially desirable responses in self-report 
questionnaire studies was not apparent in the data. 

Hierarchical Linear Modeling to Model Change 

The literacy practice change score resulting from the IRT analysis was the 
outcome variable for the analysis. As a result of our sampling procedures, 
participants were nested within classrooms, meaning that there were multiple 
respondents to the questionnaire per classroom. Because we were examining 
instructional effects on the outcome measure, each classroom was expected to have a 
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unique effect on change in literacy practices. Thus, if we used ordinary estimation 
methods and ignored classrooms’ effects, the errors would not be independent (Bryk 
& Raudenbush, 1992). A preliminary analysis showed that the intraclass correlation 
(a measure of dependence among observations) was 0.46, indicating that 
approximately 46 percent of the variance in the outcome measure was attributable to 
classroom-level differences. Even a small intraclass correlation coefficient can 
inflate error rates up to seven times (Weinfurt, 1995). 

A typical method for addressing this problem is to include class membership 
in the model as a fixed effect, thus accounting for the source of dependence among 
errors. However, because there was great variability in the number of students in 
each class, and to conserve degrees of freedom for testing the effects of interest, a 
hierarchical linear model (HLM) was estimated. Like ANCOVA, HLM accounts for 
the nested nature of the data, but can handle unbalanced cell-sizes effectively (Bryk 
& Raudenbush, 1992). HLM estimates the variance of the class effects rather than 
estimating the individual effects of each class directly. This expends only one degree 
of freedom, reserving the remaining degrees of freedom for tests of the effects of 
interest (Bryk & Raudenbush, 1992). This approach was appropriate for this study 
because individual classroom effects were not of interest, but they did need to be 
controlled for. HLM also adjusts the degrees of freedom used to test class level 
effects, thus providing more accurate, uninflated tests of their significance (Bryk & 
Raudenbush, 1992). All models were estimated using the HLM for Windows 
(version 4.0) computer program by Raudenbush, Bryk, and Congdon (1998). 

To account for the varying degrees of uncertainty in the estimation of the 
outcome variable, the inverse of the standard errors were used as weights. The 
inverse of the square of the standard errors were used as weights because the weights 
are assumed to be measures of certainty rather than measures of uncertainty. Thus, a 
weighted HLM model of change in literacy practice engagement was estimated. 

Respondent-Level Predictors of Change 

Change in literacy practice engagement was assumed to be related to various 
predictors, both at the respondent level and at the classroom level. These predictors 
can be thought of as possible confounding variables. Therefore, we needed to 
explore each one’s relation to our outcome variable of change in literacy practice 
engagement in order to control for them in our analysis of whether or not type of 
instruction is related to change in literacy practice. The participant-level predictors 
investigated were the following: 
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Attend: Attendance level 
LitLev: Incoming literacy level 
Gender: Gender of the respondent 
ESOL: English for Speakers of Other Languages status 

Tutor: Whether the respondent had a one-on-one tutor 

Days: The number of days from the respondent’s entry in the current adult 

education class to the time of the first administration of the survey 



Table 5 

Descriptive Statistics of Categorical Respondent-Level Variables 







Original sample 


Final 


sample 


Variable 


Values 


N 


Percent 


N 


Percent 




1.0 - Rarely 


8 


4.6 


1 


4.4 




2.0 - Sometimes 


12 


6.9 


12 


7.5 




3.0 - More often than not 


20 


11.6 


18 


11.3 


Attend 


3.5 - Between 3 and 4 


2 


1.2 


2 


1.3 


4.0 - Often 


32 


18.5 


29 


18.2 




4.5 - Between 4 and 5 


1 


0.6 


1 


0.6 




5.0 - Very regularly 


98 


56.6 


90 


56.6 




Total 


1 73 


100.0 


159 


100.0 




1.0 - Preliterate 


33 


19.1 


33 


20.8 




2.0 - First to third grade 


34 


19.7 


30 


18.9 




3.0 - Fourth to seventh grade 


54 


31.2 


49 


30.8 


LitLev 


4.0 - Eighth to tenth grade 


38 


22.0 


35 


22.0 




4.5 - Between 4 and 5 


1 


0.6 


1 


0.6 




5.0 - Eleventh grade and up 


13 


7.5 


11 


6.9 




Total 


1 73 


100.0 


159 


100.0 




Male 


49 


28.3 


46 


28.9 


Gender 


Female 


124 


71.7 


113 


71.1 




Total 


173 


100.0 


159 


100.0 




No 


100 


57.8 


98 


61.6 


ESOL 


Yes 


73 


42.2 


61 


38.4 




Total 


173 


100.0 


159 


100.0 




No 


129 


74.6 


118 


74.2 


Tutor 


Yes 


44 


25.4 


41 


25.8 




Total 


173 


100.0 


159 


100.0 



The descriptive statistics of the categorical variables are presented in Table 5, and the 
descriptive statistics of the quantitative variables are presented in Table 6, using both 
the entire sample (N=173) and the reduced sample for whom an estimate of change 
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in literacy practice engagement was available (N=l 59). Note that Attend and LitLev 
are presented in both tables since they can be considered both categorical and 
quantitative. 

Table 6 

Descriptive Statistics of Quantitative. Respondent-Level Variables 







Original sample 




Final sample 


Variable 


N 


Mean 


St. Dev. 


N 


Mean 


St. Dev. 


Attend 


173 


4.2 


1.2 


159 


4.2 


1.2 


LitLev 


173 


2.8 


1.2 


159 


2.8 


1.2 


Days 


171 


417.9 


506.7 


157 


435.2 


521.4 



Note that in Tables 5 and 6, the descriptive statistics changed very slightly from the 
full sample to the reduced sample with change scores. This suggests that the effect 
of losing the 8 percent of the sample should not have much effect on the outcome. 

The predictor Days was highly positively skewed. This could result in 
inordinate influence of outliers on the model estimation. To account for this 
possibility, a new variable, LogDays, was created by taking the natural log of the 
Days predictor. This new predictor was approximately normally distributed. All 
models were estimated twice, once with Days and once with LogDays as a predictor. 
The skewness of the original predictor had little effect on the estimation of the 
models. To ease the interpretation of the final model, results of models estimated 
using the original Days predictor are reported. 

The variable literacy practice engagement was to be used as a predictor at the 
respondent level as well. However, as mentioned above, literacy practice 
engagement could not be estimated in such a way as to correspond with change in 
literacy practice engagement. To investigate the effect of losing this scale as a 
covariate in predicting change, an HLM model of change was estimated using only 
first administration data. This was done because the Literacy Practice Engagement 
Scale had acceptable properties when created using only first administration data. In 
modeling first administration (of questionnaire) reported change in literacy practice 
engagement, we discovered that incoming literacy level (LitLev) was col linear with 
literacy practice engagement, and was a much more powerful predictor of change. 
Thus, we conclude that the loss of literacy practice engagement as a co variate had no 
negative effect on the analysis. 
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Classroom-Level Predictors of Chanee. 

In addition to our variables of interest, degree of authenticity and degree of 
collaboration, one could argue that other classroom-level factors could affect change 
in literacy practices. Since these could be seen as confounding variables, we needed 
to examine each for their relationship to our outcome of literacy practice change, 
allowing us to control for them in the analysis. The following classroom-level 
predictors were investigated: 






Frequency: 

Authenticity: 

Collaboration: 

ESOL: 

ABE: 

FamLit: 



Hours of class per week 

Authentic nature of the classroom 

Collaborative nature of the classroom 

Whether the classroom followed the ESOL format 

Whether the classroom followed the ABE format 

Whether the classroom followed the family literacy format 



There were also some GED classrooms, but not enough to estimate the effect of the 
GED format. Descriptive statistics of the categorical variables are presented in Table 
7, and descriptive statistics for the quantitative variables are presented in Table 8. 
Because authenticity and collaboration were the focus of this research, the 
relationship between these two variables is presented in Table 9, using both the 
original sample and final sample of classrooms. 
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Table 7 

Descriptive Statistics of Categorical Classroom-Level Variables 



Original Sample Final Sample 



Variable 


Values 


N 


Percent 


N 


Percent 




1 .0 - Highly school-only 


14 


16.9 


13 


16.9 




2.0 - Somewhat school-only 


44 


53.0 


40 


51.9 


Authenticity 


3.0 - Somewhat authentic 


22 


26.5 


21 


27.3 




4.0 - Highly authentic 


3 


3.6 


3 


3.9 




Total 


83 


100.0 


77 


100.0 




1 .0 - Highly teacher-directed 


27 


32.5 


23 


29.9 




2.0 - Somewhat teacher-directed 


41 


49.4 


39 


50.6 


Collaboration 


3.0 - Somewhat collaborative 


12 


14.5 


12 


15.6 




4.0 - Highly collaborative 


3 


3.6 


3 


3.9 




Total 


83 


100.0 


77 


100.0 




No 


74 


89.2 


69 


89.6 


ESOL 


Yes 


9 


10.8 


8 


10.4 




Total 


83 


100.0 


77 


100.0 




No 


23 


27.7 


22 


28.6 


ABE 


Yes 


60 


72.3 


55 


71.4 




Total 


83 


100.0 


77 


100.0 




No 


74 


89.2 


68 


88.3 


FamLit 


Yes 


9 


10.8 


9 


11.7 




Total 


83 


100.0 


77 


100.0 



Table 8 

Descriptive Statistics of Quantitative Classroom-Level Variables 
















Variable 




Original Sample 




Final Sample 


N 


Mean 


St. Dev. 


N 


Mean 


St. Dev. 


Frequency 


83 


7.1 


5.9 


11 


7.0 


6.0 


Authenticity 


83 


2.2 


0.7 


11 


2.2 


0.8 


Collaboration 


83 


1.9 


0.8 


11 


1.9 


0.8 
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Table 9 

Relationship Between Authenticity and Collaboration for the Final Sample; r = 0,709. 







Collaboration 








Highly 

Teacher-Directed 


Somewhat 

Teacher-Directed 


Somewhat 

Collaborative 


Highly 

Collaborative 


Highly 

Authentic 






1 


2 


Somewhat 

Authentic 


1 


10 


9 


1 


Somewhat 

School-Only 


11 


27 


2 




Highly 

School-Only 


11 


2 







Modeling Unconditional and Respondent-Level Model 

To begin the modeling process, an “unconditional model” was estimated, in 
this model, only the overall mean change score (the intercept), and the variance of 
classroom effects were estimated. For this model, the percent of variation within and 
between classrooms was calculated. 

Using the estimate (obtained from the unconditional model) of the variation 
within classrooms as a baseline comparison, a respondent-level regression model was 
estimated. The respondent-level predictors were entered into the respondent-level 
model one at a time and then together. To determine whether these predictors should 
remain in the model, five criteria were considered. 

First, the significance level of each respondent-level predictor’s regression 
weight was considered. When statistical significance was observed, this provided 
evidence that the predictor should remain in the model by showing that differing 
levels of the predictor predicted statistically significant differences in the outcome. 

Second, the significance level of the variance of each respondent-level 
predictor’s regression weight across classrooms was considered. When statistical 
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significance was observed, this also provided evidence that the predictor should stay 
in the model by showing that the predictor’s regression weight varied in a statistically 
significant manner from classroom to classroom. 

Third, the unique percent increase in respondent-level variation explained by 
each respondent-level predictor was considered. By comparing models with new 
predictors to the most predictive model nested within the new model, an estimate of 
the unique contribution of each predictor could be obtained. The nested model used 
for comparison always contained all predictors from the new model except the new 
predictor. 

Fourth, the significance levels of chi-square difference statistics were 
considered. Each model estimated in HLM produced a measure of deviance that is 
distributed as a chi-square statistic with the same degrees of freedom as the number 
of variances estimated in the model. Similar to the third step, new models were 
compared with the least deviant nested model by calculating a chi-square difference 
statistic distributed as a chi-square with degrees of freedom equal to the difference in 
the degrees of freedom of the nested models. The statistical significance of this test 
was interpreted as the statistical importance of the unique contribution of the new 
predictor. 

Fifth, the collinearity of each respondent-level predictor with all other 
predictors was examined. The standard errors of each predictor in each model were 
recorded and compared across all models in which they occurred. When standard 
errors became inflated, this was taken as a sign of collinearity with one or more of 
the other predictors in the model. 



Modeling Classroom-Level Models 

The final respondent-level model was used as a baseline comparison for 
models including classroom-level effects. First, a classroom-level model of the 
respondent-level intercept (or base model) was developed. Next, classroom-level 
models of the respondent-level regression slopes were investigated. 

Using the estimated classroom variation in intercepts as a comparison, the 
base model was developed. The classroom-level predictors were entered into the 
classroom-level model one at a time and then together. To determine whether these 
predictors should remain in the model, four criteria were considered: 
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First, the significance level of each classroom-level predictor’s regression 
weight was considered. When statistical significance was observed, this provided 
evidence that the predictor should remain in the model by showing that differing 
levels of the predictor predicted statistically significant differences in the outcome. 
Second, the unique percent increase in classroom-level variation in the intercept 
explained by each classroom-level predictor was considered. This criterion is 
analogous to the third criterion for the respondent-level model. Third, the 
significance levels of chi-square difference statistics were considered. This criterion 
is analogous to the fourth criterion for the respondent-level model. Fourth, the 
collinearity of each classroom-level predictor with all other predictors (at both the 
respondent and classroom level) was examined. This criterion is analogous to the 
fifth criterion for the respondent-level model. Models of the respondent-level 
regression slopes were developed in the same way as the base model. 

Assessing the Assumptions of HLM 

Four of the assumptions of HLM can be assessed directly. All other 
assumptions are logical exercises that must be addressed theoretically. The four 
assumptions that can be addressed empirically are described below. Note that all 
assumptions may be checked at each stage of model development, but in this case, 
only the final model was checked for violations of the assumptions. 

The normality of the respondent-level residuals was checked by inspecting 
histograms and normal Q-Q plots of the respondent-level residuals. Homogeneity of 
the respondent-level residuals (or difference between observed and predicted values) 
within each of the classroom units for the difference between observed and predicted 
values was tested with each run of the models. 

The normality of the residual variation in classroom-level prediction of 
respondent-level intercepts and slopes was checked individually for each intercept 
and slope by inspecting histograms and normal Q-Q plots of classroom level 
residuals. It was also checked in the multivariate by inspecting the Mahalanobis 
distance of the residuals of each classroom with the intercept and slope against a chi- 
square distribution. This was only done for classroom-level predictors that varied 
across classrooms, since the variance of these effects were estimated only for these 
variables. 

Homogeneity of the residual variation in classroom-level prediction of 
respondent-level intercepts and slopes across all values of the classroom-level 
predictors was checked by inspecting scatter plots of classroom-level predicted 
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values versus residuals for both the intercept and the slopes being predicted. This 
was also only done for classroom-level predictors that varied across classrooms. 
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CHAPTER SIX: RESULTS 
Final Model 

The final respondent-level model included three respondent-level predictors: 
(a) literacy level of the student upon entry to the class (LitLev), (b) whether or not the 
student was in an English-for-speakers of-other-Languages class ( ESOL ), and (c) the 
total number of days the student had attended the class prior to the first 
administration of the home literacy practices questionnaire (Days). These were the 
three participant-level variables that showed statistically significant relationships to 
the outcome variable of change in literacy practice. The final classroom-level model 
of the respondent-level intercepts included two classroom-level predictors: the 
degree of authenticity of materials and activities in the class (Authenticity) and the 
ABE status of the class (ABE). These were the two classroom-level variables that 
showed statistically significant relationships to change in literacy practice. No 
interaction terms were statistically significant either at the respondent level, the 
classroom level, or across levels. 

The final model was estimated using the natural and log days metric, and 
using weighted and unweighted change scores in all combinations, producing four 
possible final models. Table 10 presents the parameter estimates of the HLM 
analyses for each of the four methods for estimating the final model. 

As seen in Table 10, LitLev and Days are the most powerful predictors of 
change in literacy practice engagement, since they have the largest effect sizes (from 
-.308 to -.379 for literacy level and from .207 to .309 for days attended class). The 
lower the student’s literacy level upon entry to the class, the more likely he/she was 
to report change in literacy practices. Further, the longer students had been attending 
their class, the more likely they would report change in their literacy practices. Both 
of these predictors were moderately related to change in literacy practices. The third 
respondent-level predictor, whether or not a student is an ESOL student, was 
negatively related to change in literacy practices. Regarding classroom-level effects, 
students who were in ABE classes, and in classes with greater degrees of 
authenticity of activities and materials were more likely to report change in their 
literacy practices. These two classroom-level predictors were moderately related to 
reported change in literacy practices, as indicated by their effect sizes. 
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Table 10 

Parameter Estimates 







Log days metric 










Natural days metric 








Weighted 




Unweighted 




Weighted 




Unweighted 


Parameter b* 


b** 


P 


b* 


b ** 


P 


b* 


b** 


P 


b* 


b** 


P 


Intercept -.553 


.043 


.116 


-.642 


-.030 


.045 


.393 


.043 


.079 


.090 


.002 


.770 


LitLev -.263 


-.308 


.000 


-.354 


-.415 


.000 


-.029 


-.337 


.000 


-.323 


-.379 


.000 


Days .204 


.240 


.001 


.262 


.309 


.000 


.000 


.207 


.000 


.001 


.256 


.000 


ESOL -.557 


-.268 


■ .000 


-.565 


-.271 


.000 


-.500 


-.247 


.001 


-.355 


-.170 


.045 


Authent .204 


.147 


.021 


.185 


.134 


.043 


.217 


.162 


.009 


.204 


.148 


.023 


ABE 


- 


ns 


- 


- 


ns 


- 


- 


ns 


.387 


.169 


.036 



* unstandardized coefficient 
** standardized coefficient (or effect size) 
ns not significant 



Note that the effect of ABE was statistically significant only when using 
unweighted change scores being predicted by the natural days metric (the least 
sophisticated modeling method). In addition, the significance level of ESOL varied 
across the four modeling methods, with increases in significance when using more 
sophisticated methods. 

As mentioned above, HLM adjusts the degrees of freedom used to test the 
classroom-level effects to produce uninflated significance tests. Because of this, the 
effective sample size for testing those effects is the number of classrooms in the 
study (77). Even with this moderate sample size, the effect of classroom authenticity 
is statistically significant. 

The residual variation between and within classrooms is estimated with each 
run of an HLM model. The residual variation between classrooms is tested for 
statistical significance with each run of an HLM model. In all methods of estimating 
the final model, the residual variation between classes was statistically significant, 
supporting the use of HLM methodology. 

Table 1 1 presents the estimated residual within and between classroom 
variance components for the four models presented in Table 10. In Table 1 1, the 
variance components of the unconditional model, the base model, and the final 
model are presented for each of the four methods of estimating the models. These 
are included to allow us to interpret various representations of the percent variation 
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explained within and between classrooms. The variance components displayed in 
Table 1 1 are the residual variation both within and between classrooms that is 
unexplained by effects contained in the model. Note that the unconditional model 
does include classrooms as a random effect. This model is called the unconditional 
model because we didn’t really care about classroom effects — we just wanted to 
control for them. The bottom two lines of Table 1 1 address the percent of residual 
variation unexplained by the model that occurs within and between classrooms. 

Table 12 uses the information in Table 1 1 to calculate the percent variation 
explained within and between classrooms in different ways. When proceeding from 
the unconditional model to the base model by incorporating respondent-level 
variables, Table 1 1 shows that residual variation both within and between classrooms 
is reduced. This is mirrored in lines 1 and 4 of Table 12, respectively. Thus, 
respondent-level variables explain variation not only at the respondent level, but also 
at the classroom level. However, a comparison of lines 1 and 4 show that 
respondent-level variables were relatively weak in explaining within classroom 
differences, but quite powerful in explaining between classroom differences. This 
suggests that values of the respondent-level variables varied systematically by 
classroom, thus explaining some of the classroom level differences in the outcome by 
using classroom differences in the respondent-level predictors. To check this 
interpretation, a classroom-level model using classroom means of the respondent- 
level variables was estimated. This model provided very similar estimates of the 
reduction in variance both within and between classes. 

Table 1 1 

Estimated Variance Components of the Models Estimated Four Wavs. 



Source of residual variation 


Log days metric 








Natural days metric 








Weighted 


Unweighted 


Weighted 


Unweighted 


IT 


B r F* 


U’ 


B* 


F* 


U* 


B T 


F* 


U’ 


B t 


F* 


Respondent (within class) 


0.262 


0.231 0.237 


0.574 


0.480 


0.487 


0.262 


0.250 


0.242 


0.574 


0.520 


0.524 


Classroom (between class) 


0.469 


0.253 0.222 


0.475 


0.163 


0.139 


0.469 


0.225 


0.175 


0.475 


0.128 


0.087 


Percent within classrooms 


36 


48 52 


55 


75 


78 


36 


53 


58 


55 


80 


86 


Percent between classrooms 


64 


52 48 


45 


25 


22 


64 


47 


42 


45 


20 


14 



Unconditional model 
f base model 
* final model 



Table 12 



54 




63 



NCSALL Reports #17 



November 2000 



Percent Variation Explained of Models Estimated Four Wavs 







Percent variation explained 






Log days metric 


Natural days metric 


# Type of variation explained 


Weigh te 
d 


Unweighted 


Weighted Unweighted 


1 Unconditional variation within classrooms explained by base 


12 


16 


4 


9 


2 Unconditional variation within classrooms explained by final 


9 


15 


7 


9 


3 Base variation within classrooms explained by final 


-3 


-1 


3 


-1 


4 Unconditional variation between classrooms explained by base 


46 


66 


52 


73 


5 Unconditional variation between classrooms explained by final 


53 


71 


63 


82 


6 Base variation between classrooms explained by final 


12 


15 


22 


32 


7 Total unconditional variation explained by base 


34 


39 


35 


38 


8 Total unconditional variation explained by final 


37 


40 


43 


42 


9 Total base variation explained by final 


5 


3 


12 


6 



When proceeding from the base model to the final model by incorporating 
classroom-level variables, Table 1 1 shows that the residual variation within 
classrooms remains essentially unchanged, but the residual variation between 
classrooms decreases. This is mirrored by lines 3 and 6 of Table 12, respectively. 
Line 6 shows that after controlling for differences in respondent-level predictors 
(including both individual differences and systematic classroom differences), from 
12 to 32 percent of the remaining classroom variation was explained by adding ABE 
and Authenticity to the model. Thus, the respondent-level characteristics in this 
study are much more predictive of differences in the outcome than are the classroom- 
level characteristics in this study. In other words, literacy level of the student at entry 
to her class and the number of days she had attended the class were more strongly 
related to change in literacy practices than the degree of authenticity in the class 
instruction. Despite this, degree of authenticity of literacy instruction was still 
significantly related to change in literacy practices after controlling for these 
variables. 

All four methods of estimating the final model satisfied all assumptions of 
HLM. The residuals, or differences between predicted and observed values, were 
approximately normally distributed, and were unrelated to the predicted values. In 
addition, the assumption of homoscedasticity, or equal variance with classrooms, 
could not be rejected. 
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Summary of Results 

Regarding our research questions, the results show that, controlling for all 
other variables in the final model, the degree of authenticity in the activities and 
materials used in adult literacy instruction was significantly related to the likelihood 
that adult literacy students in those classes will report change in frequency and/or 
type of out-of-school literacy practices. The effect size of this relationship varied 
from .134 in the logdays unweighted model to .162 in the natural days weighted 
model. This is considered a small to moderate effect in social science research. On 
the other hand, there was no statistical effect of the degree of collaboration between 
student and teacher on reported change in literacy practices. Finally, there were no 
statistically significant interaction effects, meaning that all statistically significant 
effects were simple, easily interpretable main effects. 

Beyond our research questions, the results of this analysis provide further 
insights into the dynamics of literacy practice change. Students were more likely to 
indicate changes in their literacy practices if they attended ABE classes. On an 
individual level, the lower a student’s literacy level when beginning an adult literacy 
class and the longer the student had been attending the class, the more likely the 
student would report change in literacy practices. Finally, ESOL students were less 
likely to report literacy practice change. 
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CHAPTER SEVEN: DISCUSSION 
Limitations 

This study represents the first time anyone has attempted to examine literacy 
practice as an outcome of different forms of instruction with the adult basic 
education population. All of the difficulties of conducting systematic empirical 
research with adult education students also plagued this study, and the limitations 
thus produced must be acknowledged. First of all, all of the participants — 
teachers/classes and students — were volunteers and not randomly selected. 7 Thus we 
cannot claim true representation. 

Mitigating this, however, is the fact that the pattern of class types, 
considering the two dimensions of authenticity and collaboration, is remarkably 
similar to that found with the earlier typology study (Purcell-Gates, Degener, & 
Jacobson, 1998) which, while also relying on a volunteer sample, included many 
more programs. We worked very hard to locate more classes in the 
authentic/collaborative quadrant to better answer our research question but were 
ultimately unsuccessful. Often classes would “sound” collaborative, only to prove 
not so after direct observation and teacher and student interviews. 

A more serious threat to the issue of representation lies in the volunteer 
nature of the student participant pool. We acknowledge that students who are willing 
to accept researchers into their homes and to take the considerable time necessary to 
respond to the questionnaire items may very well be more motivated and involved in 
their literacy learning that those who are not. We take comfort in the fact, however, 
that these behavioral tendencies did not seem to be related to the type of classes in 
which they were participating, thus posing a reduced risk to interpretation of the 
results. 



Readers must remember that this study was correlational and not 
experimental. Thus, we do not claim a firm statistical causal connection between 
particular types of literacy instruction and changes in literacy practices. However, 
because we controlled for all other influencing factors identified in the data, we can 
conclude tentative causality based on the logic of causal-comparative research 



7 Most of the surveys were sent to randomly selected programs, though the decision to respond was 
voluntary. 
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(Hittleman & Simon, 1997). We believe that this type of design is close to being the 
best that researchers can do with this population of adult learners. Good 
experimentation requires random selection of program and random assignment of 
student to program. Adult students vary widely in their decisions to participate in 
adult education, how long to participate, motivations for participation, and so on. 
This variability reflects the diverse personalities, histories, life trajectories, and so on 
of adult students. It would be difficult, and probably impossible, to conduct a good 
experimental study within this context with a high enough N to produce valid and 
reliable results. 



Dimension of Student/Teacher Collaboration 

We failed to find an effect on literacy practice change of the degree to which 
students and teachers shared power in decision-making as regards curriculum and 
program governance. Although the logic of the hypothesis still holds - that students 
will engage in literacy more in their lives if they can truly influence what they learn 
to read and write in school - we could not detect a relationship. This could very well 
be the result of the way we operationalized and documented this dimension. It could 
also be that the operative elements of this dimension cannot be measured in the ways 
we undertook. Clearly, one of the aspects of a dialogic relationship between teacher 
and student is affective in nature. While this could, and should, affect change in 
literacy practices in the long run, perhaps the best way to document this aspect is 
through ethnographic methods such as those conducted in the Purcell-Gates and 
Waterman (2000) study. Of course, it is also possible that there ]s no effect on 
literacy practice change of this dimension, and that our finding of no relationship is a 
significant one in that sense. This would not be to say that dialogic relationships 
between students and teachers are not worthy of striving for. Rather that one of the 
outcomes is not literacy practice change. 

Factors Related to Change in Literacy Practice 
Individual Student-Level Factors 



The results of our analysis provide some very real insights into the different 
instructional factors that are related to change in literacy practices by adult literacy 
students. Not surprisingly, but previously undocumented, are the relationships 
between individual student factors and literacy practice change. The factor with the 
strongest relationship to literacy practice change was that of the student’s literacy 
level at the time he/she began attending the literacy class. This makes sense in that 
the degree to which a student can read would naturally affect the types of things 
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he/she will read or the frequency of reading and writing events in that student’s life. 
So, we see what we would expect: greater impact as regards change in literacy 
practices on those students who begin with lower levels of reading ability, regardless 
of instructional factors. 

Related to this logic, we find that the second strongest determinant of literacy 
practice change is the amount of time a student has spent in a literacy class. The 
more time a student has to increase his/her reading ability, the more one is likely to 
see changes in what types of texts and/or how often that student reads and writes in 
life outside of the classroom. This finding lends added significance to efforts to 
increase students’ persistence in attending adult literacy classes (see NCSALL Report 
# 12 by Comings, Parrella, & Soricone, December, 1999) . Clearly if we can find 
ways to make it possible and attractive for students to stay in classes, despite their 
pressing life-responsibilities and their low self-esteem as previous school failures, we 
can expect to see significant outcomes that accrue from these adults entering the 
literate world as readers and writers - benefits that go beyond the individual but also 
affect future generations, as we discuss below. 

Finally, the less-clear negative impact on literacy practice change of being an 
ESOL learner is also interesting. As a result of our ongoing struggle over the course 
of data collection with obtaining useable data from ESOL students, we gained some 
real insights into perceptions of ESOL adult literacy students both by practitioners 
and by the students, themselves. 

First of all, a fascinating phenomenon emerged over the course of the study 
which can perhaps be described as a confusion, or blindness, which seems to equate 
ESOL status and literate status in the minds of many, particularly teachers. By this 
we mean that many teachers when asked if certain students were “nonliterate,” 
“somewhat literate,” or “highly literate,” would answer for ESOL students as if the 
question were being asked about being literate in English . Remember that we were 
limiting our student sample to those adults who were working to improve their 
literacy skills/levels and we wanted to exclude adults who would be literate to at 
least a high school level. Therefore, we needed to know the literate status of the 
student volunteers before we included them in the participant pool, and we relied on 
teachers a great deal for this information. The confusion on the teacher’s part of 
literate status and English literate status resulted in countless episodes where we had 
to drop certain ESOL participants when we ultimately learned that they were very 
literate in their native language and engaged in wide and frequent literacy practices in 
that language in their countries of origin. 
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Added to this difficulty was the fact that the ESOL students, themselves, 
perhaps due to the attitudes of the American teachers toward their native literacy 
status, often described themselves as non- or low literate when in fact they meant 
they could not read English very well. For many it almost appeared as if they had left 
their native language literacy behind when they left their countries of origin and 
began again as new-bom nonliterate persons. Many of the ESOL participants 
refused to even discuss what texts they read and wrote in their native language, even 
when it was apparent that we were counting those with the dual-language 
questionnaire items. Others would provide answers to some items as it pertained to 
their native literacy practices and not others. As described above, all of this resulted 
in our inability to use their questionnaire data for the larger data pool on which we 
conducted the analysis. 

Whether the resultant loss of participants and missing data contributed to the 
negative finding regarding ESOL status on our research questions is impossible to 
say. Regardless, this finding meant that ESOL students showed fewer changes in 
literacy practices after beginning their ESOL literacy classes than non-ESOL 
students. This could mean that, for those ESOL students who remained in the data 
pool - i.e., answered questions about their native literacy practices and who were not 
judged (known by the teacher) to be highly literate in their native language - already 
engaged frequently in many different types of literacy practices before they began 
trying to leam to read and write English. However, it is necessary that one 
remembers that this result is based on only a few of the many ESOL literacy students 
in U.S. adult programs. All of the difficulties inherent in getting this ESOL data 
renders this finding of questionable generalizability. 

Classroom-level Factors 



The results of this study suggest that the literacy practices of adults can 
change — in nature and/or in frequency — in response to adult literacy instruction 
that is reflective of real-life literacy practices. This finding is significant on two 
levels: First, this is the first time that research has documented this for outside-of- 
school contexts, despite the widespread belief among academics that this type of 
instruction is “best practice” and despite the considerable lip-service given to this 
principle by practitioners. It is important to have solid documentation for 
instructional beliefs to withstand the inevitable shifts in the political and policy 
winds that are always just around the comer. 

Secondly, this result is significant because it demonstrates that despite the 
growing Back-to-Basics rhetoric, involving students in real life/authentic literacy 
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activities in the classroom is significantly more related to growth and development of 
literacy practices than decontextualized skill work. Since this study controlled for 
the literacy level of students, it also suggests that there is no threshold at which this 
type of practice is inappropriate (see Freire, 1993). 

Changing the Cycle of Literacy Culture 

The significance of these results, however, goes beyond the confirmation of 
hypotheses. These results suggest, empirically for the first time, a clear path that we 
can take to change the cycle of literacy culture. As this cycle includes children and 
the ease with which they can learn to read and write in school, this has incredibly 
important, intergenerational implications. 

This study was framed by the emergent and family literacy research that has 
documented repeatedly that children’s emergent literacy knowledge is formed in 
response to the literacy cultures in which they grow up during their pre-formal- 
instruction years. Further, the nature and level of this emergent literacy knowledge is 
clearly related to children’s success in learning to read and write in school. Thus, it 
stands to reason that changing the nature of the literacy cultures in which children 
grow up will change the nature of their emergent literacy knowledge, which, in turn, 
will affect the degree of success they meet in school. Previous research has 
documented that children’s emergent literacy knowledge is differentially related to 
different aspects of their home literacy cultures (Purcell-Gates, 1996). We will 
examine how these different aspects of literacy culture appear to be affected by 
instruction that is more real-life and authentic and the implications that those changes 
would have for children. 
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Frequency of Literacy Events 

Earlier studies have documented that simply counting the number of times 
someone in the home reads or writes anything will predict the degree to which young 
children in that home know that print is meaningful and that it functions purposively 
in people’s lives (Purcell-Gates, 1996; Purcell-Gates & Dahl, 1991). This concept is 
considered the very basic one for literacy development, and Purcell-Gates and Dahl, 
who termed it “The Big Picture,” found that the degree to which children possessed 
this at the start of kindergarten significantly predicted both their degree of success at 
reading and writing at the end of first grade and the effective path they would take as 
they learned to read and write in the context of formal, skills-based instruction. 
Purcell-Gates (1995), in a close ethnography of one child from a nonliterate family, 
added to this picture by providing detailed description of how the lack of this concept 
prevented the child from learning to read in school. In all of these studies, children 
who grew up in homes where people did relatively little reading and writing were at a 
considerable disadvantage when they began formal literacy instruction. The present 
study, though, documents that adults who engage in real-life/authentic literacy 
activities in their adult education classes increase the number of literacy events in 
which they engage in their out : of-school lives. This would be an important change in 
the literacy culture cycle. 

Nature of Texts Read and Written 



The study of emergent literacy development in the context of naturally 
occurring literacy practices in homes and communities (Purcell-Gates, 1996) also 
found that children’s knowledge of key concepts of written language were related, 
not only to being read to, but also to the types of texts others in their homes read and 
wrote. The higher the level of discourse involved in these literacy practices, in terms 
of their degrees of “writtenness”(Chafe & Danielewicz, 1986), the higher the 
emergent literacy scores for children in these homes. Discourse is more written (as 
compared to oral) if its syntax is more integrated and dense, its vocabulary is more 
“literary” (and less frequent), and the distance created rhetorically between reader 
and writer is greater. In the 1996 study, this higher level of discourse was 
represented by texts such as novels, essays, magazine articles, and newspaper stories. 
The lower levels of discourse were represented by text consisting solely of words 
and clauses like that on coupons or food packaging and in comics. 

The scaling of the literacy practices that underwent change in this current 
study contributes to our understanding of the significance of the results. This scale 
allows us to identify the types of practices that underwent the greater frequencies of 
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change among the participants and those that reflected fewer reported changes. As 
we see in Table 4 in Chapter 5, the two practices that were affected most by the 
changes that occurred in literacy use by the participants were reading essays, 
compositions, and text for information (i.e., expository text) and reading books and 
stories (i.e. fiction). These types of texts contain discourse at the greatest levels of 
“writtenness” as defined and operationalized in the Purcell-Gates (1996) study of 
emergent literacy and home literacy practices. The following chain of inference can 
be made: a positive change in frequency of literacy events in the home that involve 
higher levels of discourse predicts higher emergent literacy scores for children in 
those homes and positive change in frequency of higher discourse literacy events is 
predicted by participation by adults in literacy classes that include real-life literacy 
texts and activities. Thus, we can see another important alteration in the literacy 
culture cycle. Figure 4 depicts this altered cycle as we interpret the data and the 
results of this study. 

Figure 4 . The home literacy culture/school literacy success cycle as affected by 
authentic adult literacy instruction. 
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Implications for Further Research 

Clearly, the next step for researchers is to examine this inferred altered cycle 
of intergenerational literacy more directly. This would involve a large longitudinal 
study. First, researchers would need to document a large number of adult ABE 
literacy classes, with the same triangulated procedures we used for this study, to 
obtain a pool of classes with a spread of degree to which authenticity is descriptive of 
the activities and texts. Next, from these classes, students beginning instruction who 
have young (ages 3-5) children in the home would be recruited. These students 
would need to provide data, via a version of the home literacy practices 
questionnaire, across time to document changes in literacy practices. At the same 
time, the children in each family would be tested for emergent literacy knowledge 
across time points, up to the time they ended kindergarten. They then would be 
tested at least three times per year on reading and writing ability through third grade. 
At this point, using growth modeling as an analytic method, one could test the 
hypothesis inherent in the altered literacy culture cycle: Children with parents who 
attend classes with a higher degree of authenticity in the activities and texts will 
acquire greater levels of emergent literacy knowledge as a result of increases in the 
frequencies and types of literacy practices the parents engage in and, as a result, these 
children will learn to read and write at higher levels than they would have when they 
begin formal literacy instruction in school. 

On another note, we acknowledge that focusing on specific texts written 
and/or read is only one way of studying the actualization of literacy instruction and 
that literacy practice change as seen through text types is undoubtedly not the only 
outcome of instructional dimensions like authenticity and collaboration. One 
obvious avenue to pursue is that of affective change, particularly in the students’ self- 
esteem and identity, as related to different types of adult education instruction. We 
take this topic up as it related to the merging of research lenses again at the 
conclusion of Chapter 8. 
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CHAPTER EIGHT: STUDENTS’ PERCEPTIONS OF LIFE 
AND LITERACY CHANGES 

Secondary Qualitative Analysis 

The primary objective, and ensuing design, of this study was to empirically 
explore the claim that adult literacy instruction that was more authentic and 
collaborative would positively impact change in adult students’ out-of-school literacy 
practices. The quantitative analysis, just described, served this end. However, in the 
course of the coding phase of the analysis, it became increasingly obvious to us that, 
despite what results would accrue from the quantitative analysis, the participants, 
themselves, were attributing change in literacy practice to factors other than 
dimensions of their adult literacy instruction. Although they all seemed to accept our 
apparent presupposition that the topic of literacy practices was connected in some 
way to the fact that they were attending literacy classes, few of them explicitly 
contextualized any reading and writing they did in the past, currently engaged in 
more frequently, or did for the first time with the dimensions of authenticity or 
collaboration in classroom experiences. Rather, whenever they did provide context 
for these literacy practice changes, it was more often reflective of life experiences. In 
other words, their comments regarding literacy practice change, as captured by the 
data collectors on the questionnaires, reflected the socially embedded nature of 
literacy commented upon by Street (1989) and Barton (1998), among others. Wishing 
to capture this, we initiated an additional analysis, one that was qualitative in nature. 

The data for this qualitative analysis came from the home literacy practices 
questionnaire. Recall that the questionnaires included space for recording comments 
made spontaneously by participants in response to each of the questions about their 
literacy practices (e.g., “have you ever read'a menu?”). In many cases, the recorded 
responses involved unelaborated chronological data (e.g., “Began in 1983”). 
However, a large number of completed questionnaires contained more elaborated 
comments. These comments were sometimes direct quotes from participants (e.g., 
“When I got my first job” in response to the question about when the participant 
began a certain practice). More often, these comments were data 

collectors ’summaries of students ’comments (e.g., “J began practice 6 years ago 

when her husband wanted to go out to eat.”). These two types of responses made up 
the data for the qualitative analysis. The research question for this analysis was: To 
what do adult literacy students spontaneously attribute their literacy changes? 
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Data Collection 

The home literacy practices questionnaires of the 173 study participants (a 
total of 32 1 questionnaires) were divided and analyzed by four research assistants, 
who looked for elaborated information regarding changes in participants’ literacy 
practices. Comments were treated as data for this analysis only if they provided 
additional information not captured by the quantitative analysis of the questionnaires. 
For example, the comment “Reads menus more frequently now” would not have 
been included in the database since this information would have been captured by the 
yes/no question, “Do you think you’ve read any of these items more often since you 
began attending your literacy class?” However, if a specific reason was given for the 
change in frequency, such as getting a new job or going out to eat more often, the 
comment was recorded. 

Types of changes in literacy practice that were included in this analysis 
included: (a) increase in frequency of literacy practice; (b) decrease in frequency of 
literacy practice; (c) initiation of a new practice; (d) cessation of a literacy practice. 
Note that this differs somewhat from the operationalization of “change” used for the 
quantitative analysis. For that analysis, we were only interested in the construct of 
increase of literacy practice , both in frequency within practice and in new types of 
practices. However, for this analysis, we wished to explore the ways in which, as 
perceived by the participants, life events contextualized changes in reading and 
writing - both as increases in practice(s) or as decrease in practice(s). Further, for 
this analysis, we included all changes mentioned by the participant, regardless of 
when they occurred, i.e., we did not limit our analysis to those changes that occurred 
after the student began the class they were currently attending. For ESOL students, a 
language shift regarding an existing literacy practice was not coded as a change in 
practice (e.g., when a student stopped reading menus in Spanish and began reading 
them in English, after moving to the United States). 8 Also, as for the quantitative 
analysis, the data related to literacy with children was not included. 

Of the 173 participants whose questionnaire responses were considered for 
this qualitative analysis, 1 17 had comments recorded that provided additional insight 
into their literacy changes. The questionnaires of the 56 participants that were not 
considered for this analysis either had no comments filled in at all, had comments 
that did not expand on the information captured by the quantitative analysis, or had 



8 Clearly, moving to an English-speaking country constitutes a major life event that contextualizes 
change to an English-language literacy practice. However, as with the quantitative analysis, we were 
only interested in this study with literacy practice change, not the language of the literacy practice. 
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comments that did not account for changes in literacy practices. It must be noted that 
it was up to each individual data collector to decide which comments should be 
recorded and whether or not to record comments at all. Therefore, one cannot 
conclude that the 56 participants whose questionnaires were not used for this analysis 
did not share their thoughts about their changes in literacy practices. It may simply 
be that their comments were not recorded. 

Responses regarding changes in literacy practices from the 117 participants 
were entered into a Microsoft Access database. The following information was 
collected for each piece of data: (a) Subject ID; (b) Class type; (c) ESOL status; (d) 
Date student began current class; (e) Other adult programs student has enrolled in; (f) 
Student goals for joining; (g) description of interview setting; (h) Literacy practice 
(question number); (g) Frequency of literacy practice; (h) Comments made by the 
student; (I) Was change before or after class? 

After all of this data from the questionnaires had been entered into the 
database, one of the research assistants analyzed approximately one-half of it, 
creating a preliminary list of eight codes to be used to categorize the data. Two other 
research assistants then reviewed all of the data to refine and augment the initial 
codes. They independently created a list of codes and then compared and discussed 
their lists. Initially, they agreed on 12 codes. When their codes varied, they 
discussed and resolved the differences. In the process, seven of the original eight 
codes were retained, one was dropped, and ten new codes were added. The same two 
research assistants then read through the database again, this time using the new 
codes to categorize the data. The database was reorganized, categorized according to 
each of the 17 codes, and entered into a new Access database. To check for coding 
errors, the two research assistants read through each other’s data sets to ensure that 
the data in each category had been coded and sorted correctly. Based on these re- 
readings, four items were removed from one category and placed in another. Once 
the database was ascertained to be complete and accurate, frequencies and totals were 
run. 



Results 

The results of this analysis reveal a range of life changes that carried with 
them changes in the types and frequencies of literacy practices in which the literacy 
students in this participant pool engaged. Table 13 displays the categories of change 
and the frequencies with which they were cited by the participants. Following, we 
briefly discuss each category and provide example comments that exemplify the 
ways in which literacy practices reflected the lives of the participants at given 
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moments in time. The categories are not exclusive and some clearly overlap, such as 
child-related changes and changes in family situation. 

Table 13 

Life Changes Related to Literacy Practice Changes by Adult Literacy Students: 7V = 1 17 



Type of Life Change 


n 


% of Total 


Employment 


40 


34 


Schooling 


32 


27 


Living Situation 


23 


20 


Child-Related 


22 


19 


Family Situation 


22 


19 


Marital/Relationship Status 


22 


19 


Location 


19 


16 


Student’s Health 


17 


15 


Leisure Activities 


16 


14 


Finances 


13 


11 


Technology 


10 


9 


Driving a Car 


10 


9 


Religious Practices 


9 


8 


Non-School Literacy Support 


7 


6 


Needs or Interests 


6 


5 


Family Member’s Health 


4 


3 


Citizenship Status 


2 


2 



Literacy Practice Change Related to Employment 

In a literate culture such as this one, almost all jobs require some type of 
reading and/or writing, and the participant comments reflected this influence on their 
literacy practices. Often specific jobs brought with them specific literacy tasks, and 
many of the participants began reading certain types of tasks at the point they began 
new jobs. Conversely, participants reported that they stopped reading and writing 
specific types of texts when they quit the jobs that required them. For example, 
reading instructions was commonly required with the types of jobs the literacy 
students in this study occupied. Another common practice tied to employment was 
that of reading and writing notes and memos to other employees or to themselves 
while on the job. Other types of texts often mentioned within this category included 
schedules/guides, incident reports, phone books, and print on envelopes. Several 
participants reported beginning the practice of creating and using personal 
appointment and phone books once they started working. Time-management seemed 
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to be the motivating factor for this as well as the fact that employment usually meant 
they were outside of the home and, thus, needed to carry the phone numbers of 
important people in their lives with them. 

The reading of particularly complex texts was often related to job 
requirements. One 51 -year-old man who had only finished 6 th grade, worked part- 
time managing the apartments in which he and his family lived. This position 
required that he not only read applications from people wanting an apartment but 
also that he read, and understand, apartment leases - a text we considered a 
“document.” He only began this type of reading when he took on the job of 
apartment manager. 

Seeking a new job also brought with it new literacy practices such as reading 
and filling in job applications, reading notices of job openings, reading phone books 
to call prospective employers, and so on. Once employment was obtained, these 
practices - particularly those involving applications - often ceased, and if the 
participant did not find herself on the job market again, may never have begun again. 

At times, literacy practices were reported as peripherally related to 
employment. There was the case, for example, where a 34-year-old Guatemalan 
woman reported first reading comics and cartoons during her tenure as a live-in 
maid. The home contained magazines with comics and cartoons that she would read 
on occasion. She said that she stopped this type of reading when she left that job. 
Another Guatemalan immigrant reported that she stopped reading so much print on 
home entertainment products (e.g., videos) when she started working, citing the lack 
of time (for, presumably, engaging in watching videos and TV at home). Other 
participants reported new reading and writing practices related to the money they 
earned involving texts like paychecks, bank statements, deposit forms, receipts, etc. 

Schooling Brings Changes in Literacy Practices 

Another life-change that contextualized literacy practice change for our 
participants was that of attending an adult literacy class. Twenty-seven percent of the 
participants in this analysis cited this as explanatory for a specific change in literacy 
practice. For the most part, this change involved either increase in frequency for the 
practice or the initiation of a new one. Rarely did students report stopping a practice 
once they began attending a literacy class. However, they did report stopping certain 
practices at the conclusion of a class. 
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The two main factors involved in this school-practice change relationship 
were (a) increase in literacy ability; and (b) text types encountered in school activities 
that were apparently not part of their lives before attending the class. Other factors 
included time-management texts such as personal calendars that became necessary 
once students added school to their regular activities as well as the introduction to 
U.S. -culturally specific literacy texts and practices encountered by previously low- or 
nonliterate immigrants. 

Many of the respondents in this category gave evidence that their increasing 
ability to read and write, as a result of attending an adult literacy class, accounted for 
their adoption of new “everyday” literacy activities. They cited their newly acquired 
abilities to read ads, coupons, flyers, directions, menus, phone books, schedules, 
guides, money orders, gift certificates, labels, song lyrics, signs, and tickets. Clearly, 
all of these students began adult education with very low literacy levels, and many of 
them were recent immigrants from countries and life situations where they had no 
opportunity to attend school. It was within this last group of respondents where we 
saw evidence of new literacy practices tied, not only to learning to read and write , but 
also to encountering texts for the first time - texts that did not exist in their prior 
lives and cultural contexts. For example, many recent immigrants from El Salvador 
and Guatemala had grown up in rural, poor, and marginalized communities where 
such texts as written recipes or classified ads simply did not exist. As these students 
became literate in English, they turned to these texts as the needs and opportunities to 
use them arose. 

Attendance at adult literacy classes not only brought increased literacy 
abilities but also exposure to more complex ‘school-like’ texts like novels, poetry, 
journals, and essays. The comments by participants indicated that they began reading 
and writing these types of texts — for the first time, again, or more frequently — 
because they were assigned in class. There was the very strong suggestion that if the 
students had not been assigned these texts (i.e., had not attended the class), they 
would not have engaged in the literacy practices involved in reading and writing 
them. However, while some students indicated that they had stopped certain of 
these types of practices once a particular class ended, several others said that they 
started, for example, writing in a journal as the result of a class but had continued on 
their own after the class had ended or a literacy practice was no longer assigned. 

Some students also reported beginning an everyday-type of literacy practice 
in class which then became a part of their life. For example, one student who had 
dropped out of school after 7 th grade first encountered store flyers and advertisements 
in her literacy class. At the time of her interview she reported reading these texts for 
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her own purposes on a weekly basis. Another student, who also had left school after 
completing 7 th grade, reported reading directions for the first time after she read 
directions for operating a computer in her class. She added that she continues to read 
all kinds of directions now on a frequent basis. 

Changes in Living Situations 

Becoming independent, whether this meant moving into an apartment or 
house on one’s own, or taking a job, was a major impetus for new literacy practices 
for our participants. Moving into a new living space involved reading ads for 
housing prior to moving and reading documents such as leases or mortgages at the 
time of moving. One student began reading documents at the age of 1 6 when he first 
had to sign a lease of his own. He continues this practice, reading leases at least 
twice a year. 

Becoming independent also meant reading and writing for oneself such texts 
as bills, bank statements, paychecks, and receipts. These literacy practices often 
started when students began new jobs and/or moved away from family or group- 
support living situations, i.e, when they had to do them for themselves. Other 
literacy practices that participants tied to changed living situations that resulted in 
increased independence included reading schedules and guides, reading and writing 
names and addresses, reading container text like medicine bottles and lotions, and 
reading labels and titles of household items like toys, books, articles, magazines, and 
newspapers. 

Changes in Family Roles and Responsibilities 

By far, the bulk of the reported literacy practice changes that participants tied 
to family changes involved children in some way. Many literacy practices were 
initiated at the birth of children and a few were dropped. Students reported 
beginning to read print on food containers purchased for their children, bills and 
receipts, ads, coupons, and flyers, leases for apartments, health-related posters, labels 
and names of toys, print on envelopes, books and encyclopedias, stories and poetry, 
forms and applications, and instructions following the birth of their first children. 
Similarly, the birth of a child brought with it new writing practices. Students began 
writing personal letters to family members after they had children. They started 
writing lists of responsibilities for their children, filling out forms related to public 
assistance for their children, and keeping calendars and appointment books for the 
first time for time management purposes as well as to keep track of their children’s 
appointments with health care workers, friends, and childcare providers. 
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Some literacy practices stopped at the birth of children. If parents left jobs, 
then the reading and writing they were doing at work ceased while new, child and 
home-focused practices began or increased. One woman reported that she stopped 
reading stories and poetry, a literacy practice she had engaged in for 12 years, when 
her daughter was bom. One can only speculate that this mother of a two-year-old 
could no longer find the time or energy for this type of reading. 

The onset of schooling also brought with it the reading of school 
communications, schedules, and menus for the parents in this pool of participants. 
The need to communicate with teachers also resulted in the onset of note writing to 
school personnel. These types of literacy practices were always reported as stopped 
if the children were no longer in school. Also, one woman reported that she stopped 
reading school communications and information when she lost custody of her 
children. 

Other changes in family situations aside from those related to children were 
also reported as contextualizing changes in literacy practices. One man reported that 
he stopped reading bills, writing checks, and dealing with bank statements after his 
wife took over these responsibilities six years before. Often, students would report 
beginning the practice of writing messages on greeting cards, captions on photos, or 
personal letters when members of their family moved away. Another woman 
reported an increase in the frequency with which she read signs since her daughter 
moved and she was doing more of her own driving. An older student who had 
completed 8 th grade and who was enrolled in a family literacy program reported 
reading documents for the first time when her adoption of her grandchildren was in 
progress. She has continued to read the documents giving her parental rights to 
“reassure herself that the adoption ... is final and to feel good about having gotten 
them.” Increased responsibility for family members for several of the participants 
resulted in the new practices of reading medicine/prescription bottles, tickets, print 
on envelopes, ads and coupons, and reading and writing postal letters. 

Changes in Relationships Bring Changes in Literacy Practices 

Like the birth of a first child, getting married seemed to be a watershed event 
for our participants when they reported literacy practice change, particularly the 
beginning of a practice. There was a sense of attained independence and 
responsibility, doing for oneself and another, that seemed to explain their connecting 
certain new literacy practices to the act of getting married. For example, students 
reported beginning the following practices at the time they got married: reading bills, 
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paychecks, receipts, and bank statements, reading print on medicine bottles, lotions, 
and other personal items, reading directions that go with appliances, reading family 
histories, reading at home for a job, reading labels on household items, reading 
menus, reading messages and notes, reading phone books and yellow pages, reading 
print on envelopes, reading periodicals, reading books and encyclopedias, reading 
schedules and guides, reading the lyrics in hymnals, reading print on food containers, 
reading print on home entertainment objects, writing names and addresses, writing 
checks and money orders, filling out forms and applications, writing lists, and 
reading apartment leases. 

Divorce or the death of a spouse or live-in partner also was mentioned as 
contextualizing some of these practices. Reading documents was often mentioned as 
beginning and ending with the divorce process. Increased independence and 
responsibility following the loss of a partner also appeared in reports of first-time 
writing of checks, reading directions, and schedules and guides. 

Some literacy practices were stopped when relationships changed. One man 
reported that he stopped writing checks and money orders when he got married as his 
wife took on those responsibilities. His wife had recently left him, and he reported 
that the bills were no longer being paid as he does not engage in this type of practice. 
This same man said that he also stopped writing postal letters when he got married, 
with no further explanation. Perhaps his wife took over this type of writing as well. 
A woman who had completed 7 th grade, had been in the habit of painting and writing 
captions to accompany her pictures. However, she stopped painting when her 
husband died and, thus, the caption-writing ceased also. Another woman stopped 
reading lottery tickets when she stopped purchasing them when her husband died. 

One woman’s experiences with reading menus perhaps captures the effect of 
changing relationship status on literacy practice. She reported that she first began 
reading menus when her first husband died. He did not like to go out to eat, so the 
need/opportunity to read menus had never arisen before. After he died, she began to 
go to restaurants, a practice she enjoyed. However, the frequency with which she 
reads menus is decreasing since she began a new relationship with a man who, also, 
prefers to eat at home. 



Change in Location/Changes in Literacy Practices 

Moving to a new geographical location brings with it literacy practice change 
related to local contexts that tie to literacy practices. This was especially true for 
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immigrants to the U.S., but several native-born participants also experienced literacy 
practice change in the context of a change in geographical location. Virtually all of 
the new literacy practices reported by immigrants reflected new textual sources and 
purposes that were not present for them in their countries of origin. These included 
ads and coupons, bills, bank statements, and paychecks, print on medicine bottles, 
personal items, and food containers, calendars and appointment books, captions, 
tickets, menus, signs, schedules and guides, print on home entertainment item, forms 
and applications, documents, phonebooks/yellow pages, and notes to schools or 
teachers. In addition, the need to maintain relationships with those left behind 
seemed to dictate first uses of personal letters, their envelopes, messages, and 
greeting cards by immigrant participants. 

A few literacy practices were stopped upon moving to the United States. One 
19-year-old Guatemalan woman stopped reading religious writing because she 
stopped attending mass when she immigrated. Another woman stopped buying and 
reading lottery tickets when she left El Salvador. She also stopped reading 
magazines when she came to the U.S. where she no longer purchased them. 

Changes in location within the U.S. also contextualized literacy practice 
change for native-born students. One woman with reported learning disabilities 
reported that she quit reading the guide to television shows when she moved from 
Texas to Vermont. There were more channels available to her in Texas so reading 
the guide was seen as necessary or helpful. However, in Vermont she does not need 
a guide to choose among the few channels. Several students reported writing 
personal letters to family members for the first time when brothers or sisters moved 
far away. Finally, a move to a town was cited as beginning the practice of reading 
menus for one woman who had completed 7 th grade. 

Health and Literacy Change Relationships 

Changes in the health of the students or their family members was often cited 
in relationship to a change in literacy practice. A common change reported was the 
beginning of or increase in reading associated with medicines - print on medicine 
bottles, prescriptions, and directions from doctors. Another relationship between 
health and literacy was involved in the need to read the print on food containers for 
the first time as part of the vigilance required in managing a chronic illness like 
diabetes. Several people reported increased reading of books and stories as 
connected to an illness, presumably due to increased time available for such leisure 
activities. One woman reported that she read road signs more following a car 
accident in which her son was injured. Medical reference books were discovered and 
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read by people who wished to avoid the cost of a doctor, and personal calendars and 
appointment books were begun following a health change that necessitated repeated 
doctor visits. 

Several participants reported stopping a literacy practice because of the onset 
of a health problem. One woman said that she no longer read the newspaper because 
of failing eyesight. Another woman said that problems with her eyes related to her 
diabetes caused her to stop reading as much as she used to. 

Changes in Literacy-Related Leisure Activities 

Literacy practices were begun, increased, or stopped as participants took up 
different social activities and dropped others. The buying of lottery tickets brought 
with it the reading of the print on them. When participants stopped buying the 
tickets, they stopped this literacy practice. The activity of writing to family members 
was sometimes temporary and this literacy practice thus waxed and waned. Several 
students began keeping personal journals and writing reflective pieces, often after 
being introduced to the practice in school. Sometimes, they would stop for periods 
of time and fill their days with other activities. One woman reported increased menu 
reading as she and her friends increased the amount of time they spent going out for 
lunch. This activity was relatively new to this participant and began when she started 
living alone and not cooking for herself. Song lyrics were read increasingly 
according to the take-up of activities in which songs were sung. These activities 
would often change and thus the literacy practice of reading song lyrics would 
decrease. 

Financial Practice Change 



Changes in the finances of our participants brought with them changes in 
their literacy practices. When participants had increased income, they opened bank 
accounts and thus read and wrote checks, deposit slips, and bank statements. When 
they lost jobs, they often closed bank accounts and stopped, or decreased, this type of 
reading and writing. When their finances increased, they could buy items like 
appliances and thus would read, often for the first time, the directions that came with 
them. Increased income also meant money for tickets to ball games or shows and the 
reading of these tickets. The loss of income meant the inability to buy such tickets, 
thus the stopping of that literacy practice. One woman stopped buying and writing on 
greeting cards when she lost her job and had to cut back on personal expenses. Ads 
for apartments were read when participants found themselves in need of cheaper 
living quarters. Job applications were filled out when finances dictated the need for 
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more income. Finally, one man began receiving personal letters from an aunt when 
he inherited his house. This constituted an increase in this type of reading for him. 
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Other Changes/Other Literacy Practice Changes 

The remaining categories of life changes cited by the participants in 
relationship to literacy practice change reflect this same connection between social 
activity and reading and writing. Increased or new reading of phone books/yellow 
pages or print on home entertainment items accompanied the purchase of phones or 
video players. The purchase of televisions brought the new practice of reading a 
television guide. Directions and signs were read with increased frequency when 
participants began driving cars. One woman began reading labels on items in the 
grocery store when she first obtained her driver’s license and was able to drive 
herself to a store. New or increased practices of reading religious writing and hymn 
lyrics came when participants began attending church services. Conversely, they 
stopped when other participants stopped attending church. Several participants 
reported beginning new literacy practices when they began receiving help from 
family members or good friends. For example, one woman started reading bills and 
bank statements when her father sat her down and did this with her. Others reported 
receiving help reading maps or ads and writing letters and notes from boyfriends or 
husbands. Access to, or loss of access to, the Internet brought with it concomitant 
reading practice changes. Finally, several of our immigrant participants reported 
reading documents for the first time as part of the process of obtaining U.S. 
citizenship. 



Discussion 

This analysis of participants’ spontaneously offered and captured attributions of 
change in literacy practices both illustrates the validity of a social practice frame for 
literacy (Barton & Hamilton, 1998) and contributes to this and other theories of situated 
learning and cognition (Bakhtin, 1981; Gee, 1992; Vygotsky, 1978). Barton and 
Hamilton (pg. 7), in explicating the theoretical frame of literacy as social practice, assert 
that: 



• Literacy is best understood as a set of social practices; these can be 
inferred from events which are mediated by written texts. 

• Literacies are associated with different domains of life. 

• Literacy practices are patterned by social institutions and power 
relationships, and some literacies become more dominant, visible 
and influential than others. 

• Literacy practices are purposeful and embedded in broader social 
goals and cultural practices. 
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• Literacy is historically situated. 

• Literacy practices change, and new ones are frequently acquired 
through processes of informal learning and sense making. 

As just demonstrated through this analysis, the participants in this study 
situated their changes in literacy practice within their daily lives. As events changed, 
as their roles changed, as their finances changed, as their living situations changed, 
and their interests and activities changed, so changed their literacy practices. Texts 
and purposes for reading and writing those texts were taken up, engaged in with 
greater or lesser frequencies, or dropped as part of the ongoing evolutions of the 
participants’ lives in which, according to their own perceptions and reports, literacy 
was woven throughout, mediating their social activities, both personal and 
communal. As Barton and Hamilton state, “People are active in what they do, and 
literacy practices are purposeful and embedded in broader social goals and cultural 
practices” (1998, pg. 1 1). 

Barton and Hamilton (1998) describe the ways that different literacies are 
associated with different domains of life and our data certainly demonstrate this. 
Domains are defined by Barton and Hamilton as places where activity is structured 
and defined, like home, work-place, school, church, and so on. We saw through our 
data how change within these domains brought about change in literacy practices. 
When participants started attending church, for example, they took up the texts and 
literacy practices associated with that church; when they stopped attending church, 
they dropped those practices. When children entered the lives of the participants, 
changing the nature of activity in their homes, new texts were written and read for 
new purposes associated with children. New work places brought new literacy 
practices, and changed relationships brought change in literacy practices. 

The social theory of literacy practice asserts the relationship between 
networks and roles and literacy activity. Networks are defined as describing how 
people relate within social groups, and within such networks, people take on specific 
roles and assert different identities. We certainly saw throughout our data how 
change in roles affected change in literacy practices. When roles like husband, wife, 
mother, student, worker shifted, so did literacy practices. 

It must be pointed out that this does not mean that literacy practice follows 
social activity in some sort of passive way. On the contrary, we saw many instances 
where the actual acquisition of literacy brought about life/social activity change. 
Many of our participants reported first reading a multitude of everyday texts like 
signs, labels, ads, etc. for the first time, and thus participating differently in society, 
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as the result of learning to read or to read better in an adult literacy class. So literacy 
both changes people’s lives and reflects change in people’s lives. Indeed, this fact 
justifies the taking of a life history approach to the study of embodied literacy 
practice, according to Barton and Hamilton (1998, pg. 12): 

. . .people use literacy to make changes in their lives; literacy changes 
people and people find themselves in the contemporary world of 
changing literacy practices. The literacy practices an individual 
engages with change across their lifetime, as a result of changing 
demands, available resources and people’s interests. 

Bringing Together Research Lenses 

The participants of this study, in offering their own descriptions of the 
contexts for their reported literacy practice changes, provide a participant- 
perspective, or phenomenological, lens into literacy practice change. Again, they 
very rarely referred to aspects of their adult literacy instruction when offering these 
contexts of change — the focus of the quantitative, causal-correlational part of the 
study. When they did refer to their adult literacy classes in relationship to literacy 
practice change, it was to the straightforward fact that they learned to read and write 
in class and then they could begin to read and write texts or their own purposes in 
their lives outside of class. So we have two very different ways of looking at the 
relationships between adult literacy instruction and literacy practice change. 

This does not negate the results of either analysis, in our view. First of all, 
the study was designed to examine the relationship between the two aspects of 
instruction of interest - authenticity and teacher-student collaboration - on literacy 
practice change, either beginning new practices or increasing the frequency of 
already-engaging-in practices. This design dictated the data collection and 
subsequent analysis of that data. Thus, the data available to us is most reliable and 
valid for the purposes of the causal-correlational analysis. 

However, the large amount of data reflecting the participants’ own 
contextualizations of literacy practice change that came to us as a by-product of this 
data collection is still of interest. As just discussed, the analysis of this pool of data 
reaffirms and illustrates the literacy-as-social- practice theory which fits so well with 
the dominant paradigm of situated cognition and language learning. The fact that the 
students’ failed to draw a connection between changes that occurred in their literacy 
practices after they began their classes and the types of activities and materials they 
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engaged with in those classes does not mean that the relationships detected in our 
first analysis do not exist. 

Following, therefore, we offer a possible explanation for how these two 
findings - results of two very different types of analyses on two different, but 
overlapping, pools of data - might fit together within a singular lens. Recall that the 
original study was framed by the same literacy-as- social-practice theory as that 
which so nicely captures the results of the qualitative analysis. However, it is a fact 
that studies such as the causal-correlational one we have just completed are rarely 
done by researchers who assume this social-practice theory. Rather, social-practice 
research is mainly done outside of schools in the homes and communities where 
participants live their lives. In fact, it is a presupposition of the literacy-as- 
social/cultural-practice frame that the literacy taught and valued in schools is an 
autonomous literacy which is seen (incorrectly, according to this view) as 
decontextualized skill and residing solely in people’s heads in a cognitive way rather 
than the result of and part of the social and cultural lives of people (Barton & 
Hamilton, 1998; Street, 1984). Therefore, according to this precept, research in 
literacy practice cannot be done in schools but must be done in communities. 

Much social practice research looks at students’ potential conflict with the 
decontextualized literacy of schools and suggests that claims for it’s autonomy are 
false, given the social nature of all literacies. Studies such as Heath’s (1983) look at 
what the students bring with them to school in for form of socially based literacy 
practices. We, through this study, are able to look at the reverse and see what 
students are able to take from school and actualize outside of school, in this case 
engagement with specific text types. We believe that this step is a necessary one to 
begin to explain lived outcomes of instruction. 

Our careful statistical analysis of our data indicated a moderate and very real 
relationship between the offering of authentic literacy activities and texts in school 
and increases in types and/or frequencies of literacy practices, as we defined them re 
types of texts read and written, in people’s lives outside of school. Authentic was 
defined as those texts and purposes for reading and writing them that exist in lives 
outside of school - as part of social activity in communities. Therefore, we found 
that where school and outside-of-school overlap as regards literacy there is change as 
defined as increase in literacy practice. We were, in fact, measuring the effect of 
instruction that did not view literacy as autonomous, decontextualized skill, rather as 
contextualized, situated practice. 
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The result of the synthesis of these two analyses provides us with a picture of 
instruction that can impact the lived literacy of adult students. The students, 
themselves, experience their literacy practices as they exist in life - situated and 
bound by social and cultural activity. Educators can see the instructional layer of this 
in the overlap between literacy instruction that reflects this situated nature of literacy 
practice and literacy practice as it mediates people’s lives. We believe that this 
bringing together of two research lenses, with their quantitative and qualitative 
analytic methods, provides the beginning of a much needed bridge that will take us to 
an elaborated and more explanatory picture of literacy learning and literacy practice. 
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